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ABSTRACT 

This study describes the use and value of the major 
information products and services provided by the Defense Technical 
Information Center (DTIC) . These products and services include 
technical report distribution on an on-demand basis and through the 
Automatic Document Distribution (ADD) program; secondary information 
dissemination through online searching of the Technical Reports (TR) 
database, Current Awareness Bibliographies (CAB), and Technical 
Abstract Bulletins (TAB) ; and provision of management ii; format ion 
from three managment databanks — the R&D Work Unit Information System 
(WUIS)f the R&D Program Planning Data Base (R&DPP), and the 
Independent Research and Development Data Base (IR&D). Results of 
telephone surveys of approximately 4 50 DTIC users indicate that in 
1982 there were 12.4 million readings of Department of Defense (DoD) 
technical reports identified through various channels including 
online searching of the TR database, use of CAB, and use of TAB; and 
that there were about 14, 000 users of the management databanks. 
Expressed in terms of willingness to P^y for its products and 
services, the value of DTIC to the user is indicated as $367 million, 
while its value to the f under s, expressed in terms of savings 
resulting from reading of DoD technical reports, is estimated at 
$37.5 bi llion . An executive summary, sample quest ionna i res , and a 
list of responses to open-ended question s are provided . ( ESR) 
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EXEXOTIVE SUt^lARY 



Tliis stucJy describes the use and value of the major infonration 
products and services provided by the Defense Technical Informatics Center 
(OTIC). The products and services considered include technical report 
distribution on an on demandHDasis and through the Automatic Document 
Distribution (ADD) program; secondary information dissemination through 
online searching of the Technical Reports (TR) data base. Current Awareness 
Bibliographies (CAB), and Technical Abstract Bulletins (TAB); and provision 
of management information from three managenent data banks. The amount and 
kinds of use of each of these products and services is addressed, as is the 
value associated with use. 

About 1.1 million cc5)ies of Department of Defense (DoD) technical 
reports were distributed by OTIC in 1982. The estimated 157,000 scientists 
and engineers engaged in DoD research made about 12.4 million readings of 
DoD reports, or about 80 each on the average. Most readings were for 
multiple purposes, with the most comnon purposes being application of the 
report findings to a current project and professional development, current 
awareness or general interest. On ':he average, the amount of time spent 
reading a DoD technical report is about 100 minutes. 

Technical r^^rts read are identified through various channels, 
including online searching of the TR data base^ use of CAB, and use of TAB. 
Nearly half of all DoD reports read are identified through an online 
search , with about 600,000 such searches conducted in 1982. About 12 
percent and 6 percent of all readings came through use of TAB and CAB, 
respectively. There are about 25 report readings for every biweekly copy 
of TAB issued, and about 14 readings for every CAB copy distributed. 

OTIC's managenent data bases include the R&D work Unit Infonretion 
System (WUIS) , the R&D Program Planning Data Bc^se (R&DPP) , and the Inde- 
pendent Research and Development Data Base (IR&D) . Uses of the three data 



bases in 1982 are estimated at 280,000, with about three-quairters of the 
use being of WUIS. TSiere were about 14#000 users of the management data 
banks in 1982f with an average of about 20 uses each. 

"Hie value of t3TIC's information products and services is considered 
from two perspectives; that of the user# expressed in terms of willingness 
to pay, and that of the funders# expressed in terms of savings resulting 
frofu readinj of DoD technical reports. Value is estirated at $367 million 
from the users' perspective. From the funders' perspective! value is 
estimated at $37.5 billion# 100 times the value expressed in terms of 
willingness to pay, 

'ID consider the value of the UYIC products and services relative to 
the value of substitute services, estimates were made of the repc^tt read- 
ings that would be lost if infonration expenditures were fixed and other 
available bibliographic services were used to identify DoD reports. Tlie 
value of readings lost by use of such substitutes is taken as the value of 
the OTIC products and services. The total value to the user measured in 
this way is $77 million and the value to the funder is $8.3 billion# again 
appro xirrately a hundredfold increase. 

These findings establish significant levels of value associated 
with the provision of OTIC products and services. There are substantial 
readings of DoD technical reportsi with about 12,4 million such readings 
estimated for 1982. The savir^gs value associated with an individual read- 
ing — $4,600 on the average — is high. The total value to the Department 
of Defense, reflecting only those reaoings coming through OTIC biblio- 
graphic products/ is $37.5 billion. This suggests a considerable return on 
the investment in OTIC's information products and services. 
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The value of infonr^tion services and systems to the user connmi- 
ties they are intended to serve is receiving considerable attention in 
today's clirrate of resource control and ecaiomic rationalization. Irrplicit 
in the Federal Government's provision of information products and services 
to researchers is the assumption that the use of these products and 
services will contribute, directly or indirectly, to the Government's 
mission. Questions arise, however, in attenpting to pinpoint the linkages 
between a complex information system and research activities and in deter-^ 
mining the appropriate types and levels of services to be provided. Other 
questions which arise concern how and by whom information services can best 
be made available. 

Among the agencies facing such questions is the Defense 
Department's Technical Information Center (OTIC). OTIC provides informa^ 
tion on Department of Defense (DoU) research and development activities to 
the Government and to DoD contractors, subcontractors, and grantees. A 
range of products and services are provided, including both primary and 
secondary information distributed in a number of forms. In providing these 
products and services, OTIC is concerned with the sorts of issues mentioned 
above and with the underlying question of benefits derived by their users. 
Reflecting these concerns, OTIC requested that King Research, Inc. perform 
a study to explore the value of their products and services. This report 
documents that study. 

The purpose of the study was to look at the major OTIC products and 
services, to investigate their levels of use by DoD scientists and 
engineers, and to estimate the value associated with that use. The metho^ 
dology used here has been developed by King Research to consider questionf> 
of value of infoonatioi products and services of al] types. Tti2 approach 



taken is based on the premise that the value of an informtiai product or 
service stems from its use# and from the subsequent application of informa- 
tion acquired from the product or service. V^lue is addressed at three 
levels, or from three perspectives: that of the useri tiie fun(?er (in this 
case the Department of Defense) , and of society in general. Economic 
models are used to estimate both the total value of infoniQtion provided 
through a particular source and the net value associated with providing the 
information through one particular source rather than some other. 

This approach to the measurement of value was first applied to the 
Department of Energy's Technical Information Center (TIC). Hie results of 
that study are documented in King Research's Value of the Energy Data Base 
[1] . Like OTIC, TIC provides both primary and secondary information on 
research and development activities to scientists/ engineers and others 
associated with their agency's programs. Where appropriate in this report/ 
results from the TIC study are presented for cOTparison with those from 
OTIC. 

To obtain information on the use and value of OTIC's products and 
services, existing data on distribution and on users were first analyzed. 
From this, the ccrrposition of the user coimunity was identified. Regis- 
tered OTIC users are generally organizations, representing groups of indi- 
vidual users. Actual use data came from a telephone survey of these indi- 
viduals in the user organizations/ with the individuals randomly selected 
from among research scientists and erigineers in the organization. Survey 
topics included general information use, use of OTIC products and services, 
characteristics of the lart use of a particular OTIC product or service/ 
and the value associated with that use. More informatiai on survey methods 
is provided in ^^ppendix B. 

In this study of DTIC's information products and services/ we 
looked closely at the use of information at various classification levels; 
that is/ classified/ unclassif iecVlijriited, and unclassifiecVunlimited. n^e 
purpose of this was to look at differential patterns of use and of value of 
each of these categories of material. Also of special interest in this 



stacJy was the use made of OTIC's management infonnation data banks. There 
are three such data banks; the Research and Technology work LYiit Inform- 
tion System {WUIS) , the Research and '>evelofwent Program Planning (R&DPP)1, 
and the Independent Research and Development (IR&D) • For each of these 
clata banks we lookeO again at levels of use and associated levels of value. 

1 .2 Value and Its Measursnent 

Kl£L- yalue Qf_^nfQn[aticn and Information Prndiirts and Services 

'Itirough the years our society has taken information for granted, 
just as it has many of our other abundant resources. Now with treinendous 
pressures br^ing brought to bear on budgets and other resources, <"he infor- 
mation commur.ity is faced with increased deinands to justify it,^ expendi- 
tures and, in some instances, even its existence. Unfortunately there has 
been no calculus or economic theory developed to measure the value of 
infonnation. This is partially because in the past there has been no 
pressing need to do so and partially because of the difficulty in doing so. 
The difficulty arises largely because of the inability to define or quan- 
tify infornation and to separate infornatiOTi from the product or service 
used to transmit it , In addition, the economic properties of the 
infonnation (e.g., as a public good or private good) change depending on 
where in the infonnation transfer process it resides (i.e., in the hands of 
an author, publisher, library or user). 

Information generated through public funding, such as that produced 
through Department of Defeise research and development funding, presents an 
additional set of issues. On the one hand infonnation is part of what is 
produced through R&D efforts and the infonnation 's value is partially 
defined by how much it is subsequently used. On the other hand, R&D 
efforts are heavily dependent on existing knowledge to build upon or avoid 
the necessity of regenerating existing infonnation. Conducting research 



1 R&DPP data bank activity was discontinued in December 1982. 



and development is like mnufacturing a product in the sense that it is 
dependent on a number of neterials or ingredients. Inforrotion is among 
the neterial or ingredients that contribute to research and developcr^nt 
output. The purpose of the Defense Technical Infonration Service is to 
collect! process and provide access to inforration qenerated f ran DoD 
research and development efforts. The purpose of this report is to gain 
some notion as to the value of that infonration and the contribution that 
OTIC makes i^^ providing it. 

As will be shown in the next section, scientists and engineers 
devote an enormous anount of time to acquiring, reading and using infomB- 
tion. The question becomes one of whether one can place a value on the use 
of infoniBtion. King Research has attempted to look at the value of infor- 
HBtion from several perspectives; those of users (i.e.# the DoD funded 
scientists and engineers), the DoD funders and society. The time that 
scifintists and engineers have to conduct their work is a scarce resource 
since there is so much denand on it. Presumably# they make rational deci- 
sions on how to spend that time in a way that is most beneficial to their 
work and their own self-interest. They (or their representatives such as 
librarians) also have limited funds available for purchasing infonration 
products or services so that decisions to acquire these should also be made 
on a rational basis. From the perspective of scientists and engineers, one 
way to measure value ic by measuring what they are willing to pay in terms 
of the moneys exchanged or time and effort needed to identify/ locate# 
order, receive and use infonration. What they pay in these vrays is called 
effective price in this report. 

TSie value of a body of information such as that found in DoD 
reports can be measured by multiplying the average effective price times 
the number of times infomBtion is read (used). This might be considered a 
lower bound of value represented by what users are willing to pay. It is 
an underestijrate in the sense tliat some users would presumably be willing 
to pay more. Furthermore! this lower bound of value does not include the 
value derived from leading infonration. It is shown iii this report that 



reading is done largely for the purposes of applying the results for 
conducting research , educat ing oneself , educating others, writ ing, 
ranagament^ etc. Each of these purposes should yield some t nefits, such 
as saving research time and materials. Tlius, additional value should be 
achieved through improved research results, education, writing, and 
TOnageT>ent . 

Considered from the perspective of scientists and engineers, value 
is measurecl in terms of the user's willingness to pay as described above. 
Another perfective is how the reading and use of infonnation affect their 
work, tenifestations of the value of reading infonration are in the labor 
time saved? materials, equipment, travel supplies, telephone calls, etc. 
saved; improved research and development activities; iinproved education of 
oneself and others; iirproved decision niaking; and so oi. This might be 
thought of as the funder's perspective. A third perspective involves how 
the work of scientists or engineers affects the objectives of their 
organizations or tie goals and mission of society. Examples of such value 
might be how the work results in better national defense, improve balance 
of trade, improve quality of life, ed^jcate future scientists and engineers, 
improve the research and technology in other disciplines (through 
technology transfer), protection of sensitive information, and so on. The 
degree of difficulty in quantifying and measuring value is least with the 
sc ient ist 's or enginee r ' s per specti ve , greater with the f under ' s 
perspective, and very difficult with society's perspective. 

All of these measures of value presumably increase with greater 
frequency of use of the infonnation. Tlius, factors that led to increased 
use of infonration enhance the value of the information. Such factors 
include lowering the effective price of infonnatioi which includes reducing 
the charge for inforrnatioi products or services as well as reclucing the 
cost to the user of erploying th&)f providing better infonration products 
or services? increasing awareness of the information and :neans of getting 
it; or eliminat ing coist ra ints on availabi lity of information (i.e., 
security classification, transborder flow, restrictions through ownership 
of intellectual property, etc). 
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There are other factors as well which affect the value of 
infoniBtion. For exanplei primary infonnation is used only if the products 
or services enplc^ed to transfer it are purchased (or otherwise acquired) 
and used. The extent to which journals or technical reportsi for exairplei 
are purchased depends on a nunber of factorsi such as price; quality of 
infomution contenti graphics, fonrat, etc.; perfonrancef such as ^eed of 
delivery and currency; and awareness of the existence of the journal 
articles and technical reports. Awareness can come in nany waysi such as 
advertising, word of mouth, citationsi just browsing i and searching 
bibliographic products and services, to name a few. Similarly! use of 
secondary infomBtion comes only afher purchase or acquisition of secondary 
products and services r which in turn depends on pr ice, performance r 
quality! and awareness. 

The relationships above are shown in Figure 1, where statements of 
relationships are positive ones for illustrative purposes. For exanplei 
lowering effective pr icei increasing quality, inproving performance or 
increasing awareness of a product or service should increase purchases (or 
acquisitions). Of coorsei subscribing to a journal does not necessarily 
mean it will be read. Ha/ever i considering all subscribers one would 
expect an increased number of cabscriptions will yield an increased number 
of readings and use of primary infonnation. Infonnation is used for many 
purposes such as education of oneself and othersi researchi and management. 
Presumably each activity will be iirproved through greater use and ulti- 
mately will help the nation by inproving productivity! balance of tradei 
use of resources, quality of lifei and so on. 

InfomBtion is found in several kinds of documents! such as journal 
articles! technical reports, books, patent documents, numeric data bases, 
etc. In this study the focus is on the infonnation found in technical 
reports. An estimate is ^ade of the effective price and amount of reading 
done by DoD funded scientists and engineers. The effective price was 
estimated by conput ing user expenditures related to searching (i .e. , 
through on-line search, use of published bibliographies, browsing, etc.), 
acquiring the technical reports (i.e., through personal subscriptions or 



R&D 



Lower 
EftecTive 
Pri« 



Primary Inlorr-Btion 



Development 0^ Prrmiry 
^roducu & Service! 



Development SecOndiry 
Produni & Servicej 



Bener 
Performance 



Lower 
Htfenive 
Price 



Bener 
performance 



Higher 



Greater 
Aware ' 



More PurchtJej''AcQuiJilion SecorJary 
Produce & Services (Quantities Derranded} 



Higher 
QualiTY 



Greater 
Aware 



More of 
SecOf^dary Information 



In* 

Villi* 
to 
Search- 
ers 



More PurChflJe^/AcQuiiition of Primary 
ProduciJ & Servtcej <Quantiti« Demaf>ded) 



More Uje of Pnmary Information 



Increased 
Value to 
Readers 



Educatiori 
iSelt & Chhenl 



fietearcti 



Management 



Greater Knowledge 



Better Rejearcn/ 
Higtw Productivity 



Bener Decijiom 



Greater Knowledge 



Achieve Wltjjior^* 
of Orgaf^iiation 



Beiter 
Society 



Achieve Mijjion 
of Organiution 



Better 
Society 



Better 
Society 



Increased 
Value to 
*^ Organiza* 
tionj / 
Funderj 



Increased 
> Value to 
Society 



Figure 1. Factors Related to Increasing Value of IrifDrmation 
Source; King Research, Inc. 

7 



ERIC 



purchases/ libra ries^ colleagues, etc.) # and reading . The expenditures 
include all the labor time involved, prices paid# and costs of materials 
and equipment employed. These cost data were found through secoidary 
sources (i.e., previous studies performed, by KRI and others) and through a 
user survey. Hie extent of reading of journal articles and technical 
reports was estimated from surveys where amount of reading is associated 
with the varous methods of search and acquisition. 

The determination of value of a specific infonnation product or 
service was found directly and then by estimating the cost of substitute 
products or services. For exanple, the value of TAB was estimated by 
determining what the cost and likelihood are of various substitute methods 
of searching for technical reports. Estimates are made for the cost of 
searching by means of substitute search methods where the least expensive 
substitute is always assumed to be enployed by users. A further assumption 
is made that the total cost of searching by all users will remain about the 
same regardless of whether TAB or substitute search methods are used. Thus, 
since the substitute search methods cost more, fewer searches would be 
performed and fewer readings would take place. Hie number of searches that 
can be performed under the substitute methods at the total cost is found by 
dividing the total search costs by the average cost of searching by the 
substitute methods. The number of searches lost and, hence, readings 
denied to users by not havinc, TAB is cotrputed. 

The number of readings per search is estimated by observation . 
Hius, if fewer searches are performed, one can calculate the cotrparable 
nunfoer of readings lost because a substitute search method was enployed. 
From this, one can estimate value corrputed by the change in user's willing- 
ness (necessity) to pay higher effective prices. Readers were also asked 
to indicate whether a specific reading of an article or technical report 
yielded a savings in labor time {of any of the staff), equipmait, or sup- 
plies. This estimate of savings is, in effect/ an indication of irrproved 
productivity yielded by reading. From these estimates/ the total amount of 
savings lost because substitute search services would be enployed is com- 
puted . Tliese estimates provide a conservative estimate of the value 
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derived from savings yielded by readings resulting from TAB F-_-rches per- 
tormed under existing prices. It is a conservative estimate becau:i'e other 
benef its must also accrue . Th is analysis was performed for all 
bibliographic services provideo by OTIC and for the collective services. 

1 .3 0\'&rvi€^; of thfi Rep grt 

Ihis report addresses the distribution! use# and value of six of 
OTJC'e info rmt ion products and services: on demand technical report 
dist;ributionf autistic technical report distribution (ADD), online Tech-- 
nic^jl Reports (TR) data bank searches^ the Technical Abstract Bulletin 
(TAB) and TAB indexesi Current Awareness Bibliographies (CAB), and 
Tancigemexit inf ormat ion data bank searches. Chapter 2 describes these 
prcx3ucts and services and their interrelatiofishiE)S and indicates the level 
of discriDution of each in 1982* In Chapter 3, on OTIC use, the users 
surveyed as a part of the stLidy an^ their general infona^tiai use habits 
are described^ Chapter 3 then covers usage of the specific OTIC products 
and services # including amount of use# purpose of use# methods of 
identification and 3ccess# and usage per cc^y distributed* The value of 
EHTIC products ind servicesf from the perspective of users and funders# is 
described in Chapi:er 4* Finally, Chapter 5 sunmarizes study findings and 
discusses their implications for OTIC operations. 

■The Appendices to chis report provide additional information on the 
study. Appendix B describes the methods used in surveying DoD users# and 
Appendix C is the survey instru:T>,eits. Appendix A gives verbatim resE>onses 
recewed from users in response to questions about how their time was t;aved 
using OTIC products and other issues. 
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CHAPTER 2 



CTIC'S PRODUCTS AND SE3^VICES 



Hie Defense Technical Infonnation Center (ETIC) is the clearing- 
house for the Defense Department's collectiois of research and development 
in virtually all fields of science and technology/ involving subject 
categories ranging from Aeronautics to Zoology. OTIC, a primary level 
field activity of the Defense Logistics Agency# has the mission to exploit 
the contaits of its collectioi to answer three basic questiois related to 
the Research, Develcpment# Test# and Evaluation (RCTStE) program of the 
Departjnent of Defense (DcD) . These questiois are: Cl) What research is 
being planned? (2) What research is currently being performed? and C3) 
What results were realized by cOTpleted research? 

CTIC obtains its information from Defense and associated contractor 
researchers who are required to deposit information (both unclassified and 
classified including Secret and Restricted Data) into DTIC data bases for 
the subsequent retrieval for eligible users. There are four data bases in 
DTICf of which three are considered management infonration programs. The 
three data bases of greatest in¥>ortance to the fulfillment of OTIC's mis- 
sion are the Research and Development Program Planning CR&DPP) consisting 
of planned research; the Research and Technology Work Unit Information 
System (WUIS) containing the current research being performed; and the 
Technical Report (TR) Program which is a formally documented collectioi of 
coirpleted research. The r^aining data base available at OTIC is the 
Independent Research and Development (IR&D) . 

The Technical Report data bank provides bibliographic descriptions 
of technical reports prepared under DoD funding. The data bank coitains 
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information on over 1,1 million titles, with about 27,000 titles added in 
fiscal year 1982. As shown in Table 1, the titles added in 1982 inc]udec1 
about 3,000 that were classified, 8,000 that were unclassif ie<Vlimited, and 
16,000 that were unclassifie<Vunlimited. in addition to services from the 
Technical Reports data barik, DTIC also provide copies of the actual reports 
to its user coimunity. 

Research and development activities within the Uhited States 
GovemiTten': and their associated contractors, subcontractors, and grantees, 
with current government contracts, are eligible to receive most of the 
information fron DoD data bases located at CTIC. In additim, research and 
devel<^fiient organizations without current contracts may become eligible for 
service by a military service authorization under the Defense potential 
contractors programs , There are collections, however , which contain 
proprietary data or infonnation corrpiled for the specific purpose of DoD 
management decisions which are made available to E>efense coinponents only. 

The information collected by DTIC is provided in a variety of 
forms. Technical reports are distributed both on an on-demand basis, in 
paper form and microfiche, and autocraticalJy through the Automatic Document 
Distribution (ATO) program. AT® provides microfiche copies of newly acces^ 
sioned reports selected according to a user's subject interest. As indi^ 
Gated in Table 2, CTIC distributed over a million copies of technical 
reports in fiscal year 1982, 376,000 in response to on-dacand requests and 
722,000 through the ADD program. Both types of report distribution may be 
to libraries within the DoD organizations or direJtly to individuals. 

DoD technical reports are identified by users through a nuinber of 
different channels, several of these being CTIC products and services. The 
most frequently used method of identification involves searching of the 
online Technical Report data bank through the Defense RDT&E online system. 
Searches may either be performed by DTIC or on remote terminals linked to 
OTIC's central corrputer . The nuntoer of remote sites accessing CTIC is 
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TABLE 1 



Technical Report Data Bank Input by Level of Classification: l982 



Reports Input 



Level of Classification 
Classified 
Unclassified/limited 
Unci as si fi ed/ unlimited 



Nurrfcer 
3,169 
8,0^ 

16,089 



Percent 
12 
29 
59 



Total 



27,306 



100 



Source; Defense Technical Information Center. Smmary Managgnent Data Report . 
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TABLE 2 



Annual Distribution of PTIC Products and Services; 1982 



PY 1982 

Cnc Product or Service Distribution 



On DsTiand Technical Reiort Distribution ^ 

>tLcrofiche ' 109,OOo| 

Paper Copy 2 67,000^ 

Total 376,000 

Automatic Document Distribution (ADD) . 

Microfiche 722 000^ 



Onlirte Technical Reports Data Bank Interrogations ^ 

DTIC 214,000f 

Remote Terminals 409,000 

Total 623,000^ 

3 

Technical Abstract Bulletirt and TA3 Indexes 60,000 

Current Awareness Bibliographies (CAB) 50,000"'" 

Managetnent Information Data Bank Interrogations ^ 

OTIC 18,000^ 

Remote Terminals 51,000 

Total 69,000^ 



Sources: 1 Defense Technical Infortnatian Center Sunnary Managgnent Data Report , 

2 Estimated by King Research, Inc. based on data from DTIC's Sunmary 
Managgpent Data Report , 

^ Defense Technical Information Center estiirete. 
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expanding rapidly? in Septeinber 1981 there were 422 terminals in the net- 
work and in September 1982 there were 541 terminals. As indicated in Table 
2, there were over 600f000 interrogations of the Technical Reports data 
bank in 1982, with over 400,000 of these frcm remote terminals. 

The Technical Report data bank is also used to generate subscrip- 
tion bibliographic product Si includinq tti Technical Abstract Bulletin 
(TAB), the Technical Abstract Bulletin Indexes (TAB-I), and the Current 
Awareness Bibliajraphy (CAB) program. TAB is a biweekly listing of all new 
classified and limited scientific and technical reports received by DTIC 
within the processing cycle. It is available as a bound paper document, in 
microfiche, or on magnetic tape. As noted, TAB covers only classified and 
limited reports; unclassified/unlimited reports are announced through the 
tfetional Technical Information Services (MTIS) publication , Government 
Reports Announcements and Indexes (GRA&I). 

Issued simultaneously with TAB are the TAB Indexes, which contains 
seven indexes to the documents covered by TAB and also to unclassified/ 
unlimited DoD documents announced by the tfetional Technical Information 
Sen^ice {MTIS) . TAB Indexes are provided in paper form. The 1982 distri- 
bution of TAB and TAB Indexes was about 60,000. 

The CAB program provides a customized, automated bibliography 
service based on the recurring subject needs of OTIC users. Every two 
weeks, the user's subject interest profile is matched against infonration 
contained in newly accessioned documents and a listing of citations is 
provided in paper form. In 1982, 50,000 current awareness oibliographies 
were distributed. 

DTIC's nenagement informatioi data banks include, the Work Unit 
Information System (WUIS) , the R&D Program Planning (R&DPP) data bank, and 
the Independent Research and Development (IR&D) data base. WUIS is a 
collectioi of technically oriented sunmaries describing research and tech- 
nology project currently in progress at the work unit level, including 



ERLC 



14 



infoniBtion on what, where, when, by whom, at what costs and under what 
sponsorship research is being performed. R&DPP is a repository of program 
planning documentation at the project and task level. The IR&D data bank 
vas designed to enhance conmunica'i^ion between DoD scientists and engineers 
and their counterparts in industrial organizations. It describes technical 
prograins being performed by DoD contractors as part of their i^.dependent 
research and develc^xnent programs. 

To make infomBtion in the nanaganent infomBtion data banks avail- 
able to users, otiC provides several categories of products and services. 
The data banks are available online through the RDT&E online system, which 
again nay be searched directly from the user site or frcxn C3TIC. Approx- 
imately 70,0^:^ interrogations of the three nanageroent information data 
banks were made in 1982, including 51,000 from rarote terminals and 18,000 
by DTIC. Searches by OTIC are provided on an on-denend basis and in the 
form of recurring rQX)rts conpiled periodically according to a user profile 
and in a user -defined fomBt. 

The products and services described above are the major UTIC pro- 
ducts, and were specitxcally addressed in our study. In support of its 
mission, UTIC also provides other products and services — among than the 
DTIC Digest, the Bibliography of Bibliographies, the OTIC Retrieval and 
Indexing Terminology (DRIT) , referral services, a central registry, train- 
ing programs for online system users, tours and briefings, field represen- 
tation, and review of OTIC documents — not directly covered in this 
report. 
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CHAPTER 3 



ESTIMATED USE OF DTTIC INFORMATION, PRODUCTS At© SERVICES 



3.1 nriC Users 

OTIC's services are provided prin^rily to research and developinent 
activities within the United States Government and their associated con- 
tractors, subcc -*tractorSf and grantees. Such groups with current govern-- 
ment contracts are eligible to receive most of the infornation from DoD 
(feta bases located at DTIC, and research and development organizations 
without current contracts nay become eligible for service by a military 
service authorization under the Defense potential contractors program. 
Eligibility for services is determined by a registration process. With 
registration, organizations are provided information on the services 
offered and how to request them. Registration for DPIC services also 
assists the user in obtaining the services offered by Defense-^xDnsored 
Infonration Analysis Centers and major technical libraries. 

OTIC also provides some limited services to the general public and 
to foreign requestors. Unclassified and unlimited DoD documents are 
announced and made generally available through a contractual arrangenent 
with OTIS. For foreign requestors, OTIC provides assistance through 
correspondence and telephone contacts with the cognizant foreign embassies. 

In this study, the use of OTIC services by the primary user group 
— DoD scientists and engineers, including thOF* in the Federal government 
and those in contractor organizations — was studied. An early effort \ras 
addressed to identifying the number of organizations and individuals in 
this group. 

About three thousand organizations are currently registered with 
OTIC, including some which do not rrake use of the services provided. At 
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the end of fiscal year 1982, there were 190 subscribers to ADD, 335 sub- 
scribers to CAB, and 541 terminals in user organizations able to access the 
RDTPStE online systen. The largest group of active users was the 2,C41 
organizations provided technical reports on datand. This group includes 
both heavy and occasional users of OTIC products and services; about one 
third of the group ordered fewer than ten reports, a third ordered between 
ten and fifty, and the final third ordered more the fifty. 

The nunfoer of individuals within user organizations was estimated 
at 157,000. This is based on dividing the estimated ?l5.2 billion expended 
on DoD research activities in 1962 by an average $99,000 per researcher. 
Using this nunfcer, the average nuirber of individuals per user organiEation 
can be estimated at 78. This i& an average; the organizations vary widely 
in size. In general, it was found that user organizations with large 
nuirbers of on demand requests also had large nuirbers of individual users. 
These organiEations were surveyed more heavily in this study so that each 
individual user would have an approxiamtely equal chance of being included 
in the sample. 

Of the OTIC users surveyed, 52 percent were in Government organiEa- 
tions and 46 percent in contractor organizations. Table 3 gives the pri- 
mary work role of those surveyed. As indicated, the largest group was 
researchers, with one-quarter engaged primarily in nanagement. Small 
proportiois were primarily teachers or had some other work role (such as 
information specialist). 

3.2 Use of Journal Articles. Technical Rf^ports and Other Hatf^rials 

It is estimated that there are at least 157,000 scientists and 
engineers in 1982 engaged in research and development funded by the Depart- 
ment of Defense. This accounts for about oie-fourth of the total nunfcer of 
680,000 scientists and engineers aigaged in R&D in the United States. 
Previous studies have shown that scientists and eigineers (particularly 
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TABLE 3 



Primary Work Role of Govemnent and DOD Contxactor 
Scientists and Engineers Surveyed; 1982 



Percent of 



Primary 


DOD Scientists 


Work Role 


and Engineers 


Research 


59 


Management 


24 


Teaching 


7 


Other 


10 


Total 


100 



Source: Defense Technical Information Center and King Research, Inc. Survey of 
EfTIC Use, 1982. 
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those engaged in R&D) spend an enonnous anount of time reading. Exairples 
of results from such studies are given in Table 4. 

It is clear that scientists and engineers spend about 15 to 20 
percent of their time reading the literature; about 5 percent of their time 
is devoted to reading journal articles. Studies by King Research show that 
a significant proportion of tirne is spent reading technical reports (9 
percent in energy, 12 percent in biomedicine and 12 percent by DoD scien- 
tists and engineers) . This report is largely concerned with technical 
reports prepared and read by Department of Defense scientists and 
engineers. 

One of the objectives of this study was to determine whether scien- 
tists and engineers funded by the Department of Defense depeid as ex- 
tensively on the literature as do other researchers. In particular, the 
extent to which they use the informatioi products and services provided by 
the Defense Technical Information Service (DTIC) was addressed. 

A survey of scientists and engineers funded by the Department of 
Defense showed that these persons read a great deal and that they devote a 
substantial amount of time to this reading. These results are substanti- 
ated by other results observed in the past. The estimated average annual 
nurrber of readings by scientists and engineers is given in Table 5. 

The amount of reading by scientists and engineers funded by the 
Department of Defense appears to be about the same, as or perhaps slightly 
greater than, other scientists and engineers. In total# there are about 
12.4 million readings of technical reports prepared under DoD funding. Of 
these readings/ about 9.3 millioi were from paper cc^ies and 3.1 million 
from microfiche copies. 

Very few of the readings of technical reports are what one might 
coisider to be light reading or "just to get the idea." In facti respon- 
dents reported that 47 percent of their last readings were done with great 
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TABLE 4 



Tine Spent by Scientists and Engineers Reading Journals and 
Other Materials by Field of Science: 1960-1982 



Hours per Mondi Hours per Mondi 

Field of Science Reading Journals Reading All Literature 

Physical Sciences 9.0 

25.1 36.8 

11.7 24.3 
4.8-11.7 

Engineers 5.0 

19.1 45.1 

2.2-3.5 8.6-13.8 

Life Sciences (cancer) 8.1 

24.1 

Psychology 4.8^ 

13.9^ 27.7 

Energy 9.8 23.6 

Bicc^cal 7.6 27.3 

Departnient of Defense 10.5 29.8 

Sources: King Research, Inc.: Scientific Journals in the U.S., Biomedical Com- 
ojnication Expenditures, and Value of the Energy Data Base; Institute 
of Technology, An Operations Research Study of the DissQuination and 
Use of P^cordx^d Scientific Information (Cleveland, Ohio: Case Insti- 
tute of Technology, i960); B. Weil, '^Benefits from Researcher Use of 
the Published Literature at the Exxon Research Center," Paper presented 
at the National Infon:ation Conference and Exposition, Washington, D.C. , 
April 20, 1977; T.D. Allen, Managing the Flow of Scientific and Techno- 
logical Information (Cantoridge, Massachusetts: Massachusetts Institute 
of Technology, 1966); D.W. King, D.D. Mcltonald, and C.H. Olsen, A Survey 
of Readers, Subscribers and Authors of the Journal of the National Can- 
cer Institute (Rockville, Maryland: King Research, 1978); American Psy- 
dMlogical Association, Reports of: The Anerican Psychological Associ- 
ation's Project on Scientific Information Exchange in Psychology, Vol. 1 
(Washington, D.C.: Anerical Psychological Association, 1963); D.W. King, 
and N.K. Roderer, Systems Analysis of Scientific and Technical Communica- 
tion in the Uhited States: The Electronic Alternative to ConnmicatiDn 
Through Paper-Based Journals (Rockville, Maryland: King Research, 1978); 
A.M. Hall, P. Qague, and T.M. Aitchison, The Effect of the Use of an 
ADI Service in the Information-Gathering Habits of Scientists and Tech- 
nologists (London: Institute of Electrical Engineers, 1972), p. 000. 

Q ^ Cc«n?uted as one*half of total literature. 
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TASLE 5 



Average Annual Nuriber of Readings per Scientist or Engineer of Journal 
Articles, Technical Reports and Other Ifeterials By Department of 
Defense, Department of Ehergy and All U.S. Scientists and Engineers; 1982 



Scientists 
and Engineers 

Rmded by 
Department of 
Defense 
(1982) 



Scientists 
and Engineers 

Funded by 
Department of 
Energy* 
(1982)2 



All Scientists 
and Engineers 
(1977)3 



Journal Articles 



152 



118 



105 



Teclmical Reports 

Paper Copy (DOD) 
Microfiche (DOD) 
Non-DOD 



116 



59 
20 
37 



110 



Other Nbterials 
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Respondents were told that reading meant going beyond ttie table of con- 
tents, title and abstracts to the body of the article or technical re- 
port. 

Value of the Energy Database . Donald W. King, Jose-Marie Griffiths, Nancy 
K, Roderer, and Robert R.,V. Wiederkehr. Submitted to the Departmait of Energy 
Technical Information Center. Rockville, I^farvland; King Research, Inc., I^rch 
1982. cms No. DE32-014250) . 

3 

Scientific Journals in the United States: Their Production^ Use, and Ecoojmics . 
Donald W, Kong, Dennis D. McDonald, and Nancy K. RtxJarer. Stroudsburg, PA: 
Hutchinson Ross Publishing Coiroany, 1981. 
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care and 43 percent of the last readings were done in enough depth to pay 
atteition to the main points in the text. H^e scientists and engineers 
indicated that they spent an average of about 1 .7 hours per reading of the 
technical report^i or 132 hours on the average reading DoD technical reports 
over a year's duration. The average time spent reading a microfiche copy 
was about one hour and the average time spent reading paper ccpies was 
closer to two hours. The average tiKie (per reading) spent reading other 
materials included: non^DoD technical reports # 1.7 hours? journal 
articles/ 50 minutes; other materials/ 1.6 hours. The average time spent 
reading by Energy scientists and engineers was 1.5 hours for technical 
reports and one hour for journal articles (which was about the average 
amount of tirr^e per reading observed for all scientists and enginers) . 
Tlius, results of the DoD survey seem to be consistent with data observed 
elsewhere. 

Tl^e scientists and engineers were asked to indicate their primary 
work role within their organization. Most of them were engaged in research 
(59 percent) with the renainder involved in rranagement (24 percent), teach- 
ing (7 percent) or sane other role (10 percent). As might be expected/ the 
amount of reading of DcD technical reports varies substantially by these 
work roles. TTiose engaged primarily in teaching read the most. TTiey 
averaged 102 readings per year (most of which are from paper copies) com- 
pared to slightly over 70 readings per year by those involved in research 
and mnagement. 

In Table 6, the estimated readings per DoD-fimded scientist or 
engineer are multiplied by the number of scientists and engineers (157,000) 
to obtain the total amount of reading by the group. As shown/ there are 
over 45 million readings by the group/ with 52 percent of these being of 
journal articles/ 40 percent technical reports/ and 8 percent other materi- 
als. Over one-quarter of all readings are of DoD sponsored technical 
reports. 
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TABLE 6 



Total Readings of Joiimal Articles, Technical Reports, 
and Ot±ier Materials by DOD-Punded Scientists and Engineers: 1982 



Total Readings 
Type of Material (mil lions) 



Journal Articles 23-9 



Technical Reports 18.2 

DOD (paper copy) 9,3 

DOD (microfiche) 3-1 

Non-DOD 5,8 



Ot±ier tfeterials 3-6 



Source: Defense Technical Information Center and King Research, Inc. Survey 
of nriC Use, 1982. 
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Another indicator of information use is the time spent reading . 
Table / gives the amount of time spent per itesn and per user and the total 
tiine ^)ent annually by all users. As would be expected, the average time 
spent reading a technical report (1 .7 hours, or about 100 minutes) is 
l^rg-^r than the 50 minutes spent reading a journal article. Distinguishing 
between paper and microfiche ccpies of technical reports, more time on the 
average is ^ent reading a paper copy report. The total time ^ent by an 
individual user annually is 558 hours, about 17 percent of all work time. 
Of this total, about 35 percent is spent on journal articles, 55 percent on 
technical reports, and 10 percent an other neterials. About 37 percent of 
the DoD users time spent on reading is associated with DoD technical 
reports. 

Table 7 also gives the total time spent reading by DoD contractors. 
For all types of neterials, this is 56 million hours or about 27 , 000 person 
years annually. 

It seems clear that scientists and engineers funded by the Depart^ 
ment of Defense depend a great deal on technical reports and other publica- 
tions for their research as well as for their nenagement activities and 
thei r continued education . In the next section we describe how these 
persons identify and gain access to technical reports and the extent of the 
role played by the Defense Technical Informatioi Center in these informa^ 
tion transfer functions. Hien, in the following sectioi, we indicate the 
value of infonnation found in DoD technical reports and the value added by 
one informatioi products and services. 

3.3 Use of ETTIC Products and Services 

The mc products and services covered ^ecifically in the survey 
of users included technical reports, online searches, TAB, CAB, and the 
nenageiT>ent data banks. Online searches, TAB and CAB are each used to 
identify DoD technical reports which then provide the user with primry 
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TABLE 7 



Tir:^ Si^t by DOD-Punded Scientists and Engineers Reading journal 
Articles, Technical Reports, and Other ^^aterials: 1982 



Reading Time (Hours) 

Annua 1 
Total 

Type of >faterial Per Itgn Per User (millions) 

Journal Articles .8 127 20.0 



Technical Reports 

DOD (paper copy) 1.9 111 17.4 

DOD (microfiche) 1.1 21 3.3 

Non-DOD 1.7 62 9.7 



Other Materials ^ 5.8 

Total 1.2 358 56.2 



Source: Defense Technical Infomiation Center and King Research, Inc. Survey 
of nriC Use, 1982. 
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information on corrpleted research. The iranagOTeit data banks also provide 
primary information covering planned and ongoing research. The use made of 
each of these product categories is covered in turn below. 

As indicated earlier, DoD users read a total of 12.4 million DoD 
technical reports annually, or an average of 79 per individual. These 
ridings are looked at for users in different organizations and work roles 
in Tables 8 and 9. From Table 8, the average number of readings by 
Government users is 63, while readings by contractors average 96. The 
level of reading also varies by work role with those users identifying 
teaching as their primary work role, reading about 50 more reports, on the 
average, than researchers or managers. The average number of reports read 
by researchers and managers was about the same. 

In order to read a technical report, a user must first find out 
about, that is, identify it, and then obtain a physical copy. In some 
cases these two activities of identification and access occur simultane- 
ously, as for exanple when a report is received through some sort of stan- 
dard distribution and identified by the user as being of interest. For the 
most part, however, identification and access occur s^>arately. Tables 10 
and 11 give breakdowns of the total DoD report readings by identification 
and access channels. 

In Table 10, the identification methods shown include online 
bibliographic searches, printed indexes, CAB, citations in reports or 
journals, colleague referrals, reports routed to users, and other. The 
most frequently used of these is the online bibliographic search, which 
accounts for nearly half of all readings. For the most part these are 
searches of the Technical Report (feta bank using the RDT&E system, with 
searches performed either by the user's organizatiai or by OTIC. 

Users may also identify technical reports through TAB and CAB, two 
other OTIC products. In combination with online searches, use of these 
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TABLE 3 



Annual Niaiber of Readings of DoD Technical Reports by 
Government and DOD Contractor Scientists and Engineers: 1982 



Annual Technical 
Report Readings 

Total 

User Category Average (millions) 

Government 63 5.2 

DOD Contractor 96 .7.2 

Total 79 12. A 

TABLE 9 



Annual N\jmber of Readings of DOD Technical Reports by 
DOD-?unded Scientists and Eiigineers in Four VIork Roles: 19S2 



Annual Technical Report Readings 

User Work Itole Total 

Average (millions) 

Research 68 6.3 

Management 72 2.7 

Teaching 105 1.2 
Other 1A2 

Total 79 12. A 



Source: Defense Technical Information Center and King Research, Inc. Survey 
of nriC Use, 1982. 
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TABLE 10 



Annual Nurrber of Readings of DOD Technical Reports by Government and 
DOD Contractor Scientists and Engineers by Identification Method: 1982 



Annual Technical 
Report Readings 



Nuiber Percent of 
Identification Method (millions) Total 

Bibliographic Search 

done by Organisation 5.1 41 

Done by DTIC -9 7 

Printed Index 

TAB 1,5 12 

Other 1.0 8 

GAB (Current Awareness Bibliography) ,7 6 

Citation irt a Report or Journal 1*1 9 

Colleague Referral 1*0 8 

Routed to User -5 4 

Other 5 

Total 12*4 100 



Source: Defense Technical Infonnation Center and King Research, Inc. SiiTvey 
of DTIC Us*5, 1982. 
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TABLE 11 



Annual NKniier of Readings of DOD lechnical Reports by GDvemment and 
DOD Contractor Scientists and Engineers, by Distribcticn Method: 1982 



Annual Technical 
Report Readings 



Nurber Percent of 
Distribution Method (millions) Total 

Standard Distribution to Individual .5 4 

Standard Distributicn to Library .2 2 

Ordered by Individual .6 b 

Ordered by Library 10.2 82 

Other li 7 

Total 12.4 100 



Source: Defense Technical Information Center and King Research, Inc. Survey of 
ETIC Use, 1982. 
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ident if icat ion channels yield a total of 66 percent of all readings 
identified through DTIC products or services. Identification through other 
printed indexesi which accounts for 8 percent of all readingsi involves the 
use of NASA's STAR, HTIS's GBMl, and other publications. In total, about 
three-quarters of report identifications involve a bibliographic product or 
service, with the reraining quarter made up of identification via citation 
(9 percent) colleague referral (8 percent), routing (4 percent) and other 
means (5 percent) . 

Table 11 breaks down the total report readings by distribution 
channel. There are many ways (or channels) through which ccpies of the 
technical reports are distributed including: 

(1) Directly to individuals through standard distribution. 

(2) To individuals through libraries that obtained copies 
through standard distribution. 

(3) Directly to individuals through their ordering it from 
OTIC. 

(4) To individuals through libraries that obtained copies by 
ordering them for the individual or sonieone previously. 

(5) Itirough other means such as a colleague, the Ifetional 
Technical InforrrBtion Service, etc. 

By far the greatest proportion of readings, over 80 percent, is of 
reports ordered by libraries. In total, 87 percent of readings are of 
reports ordered on demand, 6 percent are of reports received through stan- 
dard distribution, and 7 percent are accessed through other means. 

Table 12 compares the number of readings from standard and on 
cJenand distribution with the number of copies of reports distributed via 
these means . Data on both distribution and readings apply to 1982 . 
Strictly peaking, they cannot be combined to yield the average readings 
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TABLE 12 



Estimated Readings per DOD Technical Report Copy Distributed 
by Distribution Method: 1982 



Distribution Method 



Nmber of 

Copies 
Distributed 
(thousands) 



Nunber of 
Readings 
(laillions) 



Average 
Readings 
Per Copy 



Standard Distribution 



On Demand Distribution 



All Methods 



722 



376 



1,098 



.7 
10.7 
12.4 



1 

28 
11 



Source: Defense Technical Information Center and King Research, Inc. Sxirvey 
of DnC Use, 1982. 
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per copy because not all readings of reports distributed in 1962 will occur 
in 1982 and not all readings occurring in 1982 will be of 1982 reports. 
Since the levels of distribution and reading are fairly constant over the 
years however, we can combine the data to approximate readings per copy. 
The result shows about one reading per coi^y for standard distribution (all 
microfiche) and 28 readings per copy for on demand distribution (30 percent 
microfiche and 70 percent paper copy). In total, there are about 11 read- 
ings of DoD technical reports per copy distributed. This can be conpared 
with an estimated 1.1 readings per technical report copy distributed by the 
Department of Energy and .1 readings per article copy distributed for 
scientific and technical journal articles generally. 

Report readLigs can also be categorized according to the level of 
classification of the report, that is, classified, unclassif ie<VliJi^ited, or 
unclassified/unlimited. This is done in Table 13. Almost 60 parcent of 
report readings are of unclassified and unlimited materials, with about 20 
percent each for classified and limited reports respectively. The 20 
percent of readings being of classified reports can be conpared to a sim- 
ilar percent for on demand distribution of classified reports (most read- 
ings come through on demand distribution), suggesting that the number of 
readins per cc^ distributed is similar for classified and unclassified 
reports. 

Why do DoD scientists and engineers read technical reports? Gen- 
eral categories of purposes are to support education, research, and manage- 
ment. In the survey of DoD scientists and engineers, users were given the 
opportunity to indicate one or more purposes for their last reading of a 
DoD report. On the average, users mentioned about four purposes. As shown 
in Table 14, the most frequently acknowlec3ged purposes were application of 
a report findings to a current project (75 percent) and professional 
development, current awareness or general interest (74 perc^t) . About 
half mentioned application of the report *s methodology to a current 
project, use for citation, and use in preparing some publication. Comparing 
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"MJE 13 



Annual Nurber of Readings of Classified, Uiclassified/Limited and 
Unclassified/Unlimited Technical Reports by DOD-Punded 
Scientiscs and Engineers: 1982 



Level of Report 
Classification 



Annual Technical 
Report Eteadings 



Nmber 
(millions) 



Percent of 
Total 



Classified 



2.6 



21 



UnclassifiedA^imited 



2.6 



21 



Unclassified/Unlimited 



7.2 



58 



Total 



12.4 



100 



Scoirce: Defense Technical Information Center and King Research, Inc. Sijrvey 
of nnc Use, 1982. 
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TABLE 14 



Proportion of Scientists and Engineers Rinded by the Department of Defense and 
Department of Energy Who Indicated Various Purposes of 
Reading DOD Last Read Technical Reports: 1982 



Proportion of Yes Respcjnses 

a) 



Purposes of Reading Last Read Technical Reports 
Educational 

Self For professional developcnent , cur- 
rent awareness or general interest 

Others — In preparation of a lecture or 
presentation 



Department 
of 
Defense 



74 
39 



Department 
of 
Energ y 



75 
AO 



Research 



In preparation of a research proposal 27 

To apply its findings to a current project 75 

To apply its methodology to a current project 53 
In preparation of an article, book, review or 

report 49 
As a citation in an article, book, review or 

report 50 



38 
77 
50 

50 
48 



Management 



For the planning, budgeting and management 
of research 
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AO 



Other 



Source: Defense Technical Infomation Center and King Research, Inc, Survey of 
uric Use, 1982, 
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the purposes of use of DoD reports with those of DOE reports, also shown in 
Table 14 f suggests very similar patterns of use for the two groups. 

Table 15 further describes DdD report readings by indicating the 
depth to which each report was read. As suggested by the average reading 
tiii^e of 100 minutes, most reports were read either with great care (47 
percent) or with attention to the main points (43 percent) . Ten percent of 
readings were made just to get the idea of the report. 

To look at DOD scientists' and engineers' utilization of OTIC 
bibliographic products and services in more depth, s^^arate survey sections 
addressed online bibliographic searching! use of TAB, and use of CAB. 

As indicated earlier, online searches may be performed by OTIC or 
by the user organizations, increasingly in recent years by the latter. In 
1982, as shown in Table 16f about two-thirds of the interrogations made of 
the Technical Report data bank were made by user organizations. In totali 
over eOOfOOO interrogations were made. DTIC estiirates that there arei on 
the average! about three interrogations per search. 

When a search is performed! multiple citations to rQX)rts of poten- 
tial relevance are obtained • The average search of the TR data bank, 
according to usersi yields about 170 citations* This is narrowed down by 
the user to a number of titles deemed relevant. For the average DoD useri 
the number of relevant titles per search is estiireted at 39 1 or about 
one-quarter of the titles output from the online search* 

Not all reports identified as relevant are actually obtained by 
usersf and not all reports obtained are read* For the average searchi with 
39 reports identified, 15 reports are accessed and 10 read (see Table 17). 
It is the case with DoD users, as with scientific and technical users 
generally! that the number of readings resulting from an online search is 
greater than that resulting frcm a printed index search. 
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TABLE 15 



tJurber of Readings of DOD Technical Reports by Reading Depth; 1982 



Reading Depth 



With Great Care 



Uith Attenticrt to Main Points 



Just to Get Idea 
Total 



Nunber of 
Readings 
(millions) 



5.8 
5.3 
1.3 



Percent of 
Readings 

47 

43 

10 



12.4 



100 



ERIC 



Source: Defense Technical Information Center and King Research, Inc. Survey of 
mc Use, 1982. 
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TABLE 16 



Mu±ier of Online Interrogaticms of tiie Technical Reports Data Bank By ETTIC 

and by Other Organizations : 1932 



InteiTTOgaticris of the 
Searching Organization Technical Report Data Bank 



nric 214,000 

Ranote Site 409,000 
Total 623,000 



Source: King Research, Inc. Estimate based on ETTIC's Suimary Managanent Data 
Report 
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Nunber of DOD Technical Reports Identified, Accessed and 
Read as a Result of an Online Search: 1982 



Reports Identified 
Reports Accessed 
Reports Read 



Niirber 
per Search 

38.7 
14.7 
11.8 



Percent 
of Reports 
Identified 



38 



30 



Source: Defense Technical Information Center and King Research, Inc. Survey 
of DTIC Use, 1982. 
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Online searches my be performed directly by the user or by an 
intermediary such as a librarian. In the case of searches of the TR data 
base, 95 percent of all searches were performed by sonteone other than the 
end user. Thus, when users were asked about the amount of time spent 
searchingf the bulk of the time rQX)rted was associated with making the 
search request and with reviewing search output. As shown in Table 18, a 
total of 2.4 million hours were spent on online searching. This is about 
15 hours per user or almost four hours per search. Considering only the 
items read as a result of the search, about 25 minutes were spent by the 
user for each item that was ultinately read. 

Another means of id^tifying DoD reports is through a TAB search. 
In 1982, there were an estimated 1.5 million uses of TAB, or an average of 
9.5 per user . This is 25 readings for every biweekly copy of TAB 
distributed. 

TAB and TAB Indexes are used by a gmaller proportion of the user 
population than are online searches, about 30 percent of the population as 
corrpared with 65 percent. Among users of TABf more than half had used the 
publication within the last two weeks (prc^ably the last biweekly issue). 
About half were browsing when they used TAB, and about half conducted a 
problem-related search. These data seem to suggest that TAB uses are about 
equally divided between current awareness purposes and retro^)ective sub* 
ject searches. 

The average nunfoer of items identified as relevant through a TAB 
search is about 14. As shown in Table 19, about six of these titles are 
subsequently accessed and about two are actually read. The low percentage 
of reports read is prcfcably because so many of the searches were very 
recent; users were asked to indicate completed rather than planned 
readings. 

The average tiim spent on a TAB search was reported by users as 
about 45 minutes. This is about 20 minutes per itejn read, as corrpared with 
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TABLE 18 



Time Spent on Online Reports Searches by DOD -Rnded 
Scientists and Li-'grneers: .1982 



Requesting and/or Conducting Search 



Time Soent (Hours) 



Per Search 



All Seai-ches 
(millions) 



Reviewing Search Output 



3.3 



2.1 



Soiircc: Defense Technical Infomation Center and King Research, Inc. Siirvey 
of ETIC Use, 1982. 
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TABLE 19 



Nurrber of 030 Technical Reports Identified, Accessed and 
Read as a Result of a TAB Search: 1982 



Reports Identified 



Reports Accessed 



Reports Read 



Number 
per Search 

13.5 
5.6 
2.3 



Percent 
of Reports 
Identified 



41 



17 



Scfurce: Defense Technical Information Center and King Research, Inc. Survey 
of DTIC Use, 1982. 
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25 minutes for online searching. To'al time spent on TAB searching by all 
users was 1.1 million hours 

Dmc's CAB program provides a customized, autorteted bibliography 
service based on the recurring subject needs of its users. The service is 
bi-weekly. At the end of 1982, there were 335 CAB subscribers including 
both individuals and institutions. About 9 percent of DoD users, or an 
estimted 14,000 individuals, report having used CAB within the last year. 
Each of these r^>orted 22 uses on the average, for an annual total of 
314fOO0 uses. These uses resulted in about TOOiOOO readings. 

As iinplied by its title, CAB is used mostly for current awareness. 
With 26 bi-weekly copies distributed per year and 22 uses reported, it 
appears that most users are regular users. The results of an average CAB 
use are given in Table 20: about five titles identified as relevant, three 
titles accessed, and just slightly fewer titles read. In part, the high 
proportion of reports identified that are actually road is probably due to 
the broad, current awareness focus of CAB. 

For online, TAB and CAB searches, DoD users were asked to indicate 
a breakdown of reports by classification leve], . This was done for both 
reports identified as relevant and for reports read. The purpose was to 
look for different identification patterns for different classification 
levels. 

As shown in Table 21^ somewhat different breakdowns by classifica- 
tion level were observed for reports identified and reports read. More 
significant, however, were the differences between online searching, TAB 
and CAB use. ^arly half of the reports read as a result of a CAB iden- 
tification were limited, which was true of only 20 perceit of reports 
identified through TAB and 7 percent of those identified by an online 
search. Reports identified through an online search were more likely to be 
unclassified and unlimited than with either of the other two bibliographic 
products. 
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TABLE 20 



Nurrber of DOD Technical Reports Identified, Accessed and Read as a 

Result of a GAB Use: 1982 



Reports Identified 
Reports Accessed 
Reports Read 



Nurrber 
per Search 

4.5 
3.0 
2.7 



Percent 
of Reports 
Identified 



67 



60 



Source: Defense Technical Information Center and King Research, Inc. Survey 
of DTIC Use. 1982. 
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TABLE 21 



Ccpmparison of Classification Level of DOD Technical Reiwrts Identified and 
Read as a Resiilt of Online, TAB, and CAB Searches; 1982 



Identification Method and Classification Level 



Percent of 

Reports 
Identified 



Percent of 
Reports 
Read 



Online Search 

Classified Reports 
Lhclassified/Liinited Reports 
Lhclassif led/Unlimited Reports 



15 
20 
66 



30 
7 
63 



and TAB Indexes 
Classified 

Lhclassified/Limited Reports 
Lhclassified/Lhlimited Reports 



51 
U 
35 



JO 
20 
50 



CAB 



Classified 

Lhclassif ied/Limited Reports 
Lhclassif ied/Unlimited Reports 



18 
28 
54 



15 
44 
41 



Source: Defense Technical Information Center and King Research, Inc. Sxarvey of 
DTIC Use, 1982. 
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T^ble 22 corT?3ares the readings identified through HFIC biblio- 
graphic products with those from other sources. Looking at the total 
report readings from online search ing, TAB or CAB, we find 29 percent 
classified, 12 percent onclassif ie<Vlimited, and 59 percent unclassified/ 
unlimited. For other identification methods^ the proportiois are 5 percent 
classified, 38 percent onclassif ie<Vlimitedf and 57 percent onclassif ie<V 
onlimited. The bulk of readings of classified reports come from identifi- 
cations via a OTIC bibliographic products while limited rQX)rts are more 
likely to be identified by some other means. Both categories of identifi- 
cation methods are equally likely to result in rv3ading of an unclassified 
and unlimited report. 

In addition to services associated with its Technical Reports data 
bank, OTIC also maintains and provides services from its three managernent 
data banks: WUISf R&DPPf and IF&D, One hundred thirty-one users responded 
to the management databank survey conducted as a part of the project, WUIS 
users totaled 72 (or 55 percent of databank respondents) , F&DPP users 
totaled 24 (18 percent) and IF&D totaled 35 (27 percent). 

Table 23 shows the nunfoer of uses of OTIC management database by 
scientists and engineers. For the R&D Work Unit Information System (WUIS) 
the respondents' uses totaled 207,000 with an average of 1*3 uses per year. 
For the R&D Program Planning Data Base (R&DPP) there were 50^000 uses per 
year with an average of .3 uses per respondent per year. The Independent 
Research and Development Data Base (IR&D) respondents reported 23 thousand 
uses per year with an average of ^2 uses. 

The survey respondents were also asked: "Did you pass the 
infonretion you obtaind on to anyone else?" The average nunnber of pass-ons 
(Table 24) per WUIS user was 4,1, The R&DPP database users reported an 
average of 2,3 pass-ons while IF&D users averaged 3,1 passions, of 
infonretion fron a particular search. 
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DC© Technical Report Readings by Identification Method and 
Report Classification Level: 1982 



Classification I^evel 



Itac las s if ied/ Vnc las s if ied/ 
Classified Limited Unlimited 



All Readings (Millions) 2.6 2.6 7,2 

Proportion of Readliigs (%) 21 21 58 

Readings Identified through OTIC 

Bibliographic Products! (Millions) 2.4 1,0 4,8 

Proportion of Readings (%) 29 12 59 

Readings Identified through 

Other Means2 (Millions) .2 1.6 2,4 

Proportion of Readings (?□) 5 38 57 



"'" Inclijdes online searching of the TR data bank, TAB xase, and CAB use. 

Includes use of other bibliographic products, citation, colleague referral, 
routine, and other. 

Source: Defense Technical Information Center and King Research, Inc. Survey of 
DTIC Use. 1982. 
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Table 23 



Annual Number of Uses of DTIC Management Data Banks 
by DOD Scientists and Ehgineers: 1982 



Annual Uses 



Average Total 
Data Pank per year f thousands) 

R&T Work Unit 1.3 207 

R&D Program Planning .3 50 

Independent R&D _^ 23 

TOTAL 1.8 280 



Source: Defense Technical Infonnaticn Center and King Research. Inc. Survey of 
DTIC Use, 1982. 



TABLE 24 

Nucber of Pass-on Uses of DTIC Managanent Data Banks : 1982 



Average 

Data Bank Pass-On Uses 



R£J Work Unit 4.1 
R&D Program Planning 2.3 
Independent R£J) 3.1 

Total 3 . 7 

Source: Defense Technical Information Center and King Research, Inc Survev of 
DTIC Use, 1982. 
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The usefulness of DTIC management data banks was investigated 
(Table 25). Ninety-four percent of the users queried reported that the 
results of the search were useful while oily 6 percent felt that the data 
was not usefulr Of the R&DPP users, 83 percent found the results useful 
and 17 percent did not. A majority of the IR&D users also said the results 
were useful (90 percent) with only 10 percent reporting no use. Tlie 
reasons for usefulness as reported by users are listed in Appendix A. 

Corrpleteness of the information found during the user's searches 
was also measured (Table 26) . Seventy-one percent of WUIS users said "yes" 
whie 14 percent said "no" and 15 percent did not knw. Sixty-two percent 
of R&DPP users said "yesi" 17 percent said "no" and 21 percent did not 
know. Of the IR&D users# 72 percent found the information conplete while 11 
percent did not and 17 percent responded "don't know." 

Overall, nine percent of the DoD researchers and engineers surv^ed 
used OTIC's management data bases. This is about 14^000 users, not 
including those to whom the information is passed on. Since a total of 
280,000 uses are estimated, the average nuntoer of uses per user is 20. 
ttien asked about future use of the manageinent data banks, current users 
anticipated modest increases in usage over time. 
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TABLE 25 
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Rating of Usefulness of ETIC Managanent Data Banks : 1982 



Were Results Useful 
(Percent) 

Data Bank Yes No 



R^ Wbrk Unit 94 6 

R££) Program Planning 83 17 

Independent R£D 90 10 

Total 92 8 



Source: Defense Technical Information Center and King Research, Inc. Survey of 
nriC Use, 1982. 
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CTIAPTER 4 



ESTIMATED VPiLUE OF DOO INFORMATION AlB OTIC INFORMATION 
PRODUCTS AlB SERVICES 



4.1 In Terms of Readers' Wllllnqnesa to Pav for Information 

In Section 3, it is found that DoD scientists and engineers devote 
a substantial amount of their time acquiring/ reading and using infornation 
found in DoD technical reports. When this time and effort is expressed in 
terms of expenditures, it is found that they are willing to pay a great 
deal for the infornation. The estimated effective price that they actually 
pay is found in Table 26. 

Thus, the total value that DoD users are willing to pay for infor- 
mation generated by DoD is estimted to be about $560 million or an average 
of about $3f600 per scientist and engineer. The amount of this value 
directly attributable to the use of OTIC products or services (i.e., TAB, 
CP£ or computer searching) is estimated to be about $370 million or an 
average of $2#300 per scientist and engineer. 

Hiese values do not take into account that DoD scientists and 
engineers could identify and obtain their infornation in ways other than 
TAB, CAB or OTIC bibliographic searching. To obtain the effective price 
with alternate means of searchin'^, costs for the specific identification 
method (from Table 26) are added to the general purchase, processing, 
access and reading cost. For exanple, the cost of manually searching by 
other means is estimated to be $75*40 rather than $49.90 for TAB and $44.90 
for CP£, and substitition for OTIC conputer bibliographic searching is 
estimated by $51.10 instead of $43.50. (See reference [1] for details on 
how these substitutes costs are estimated) . V&lue of each of the products 
and services is estimated below by calculating the cost of changes in 
searching practices resulting from withdrawal of a product or service. It 
is assumed that a given amount of expenditures will be made by a user (or 
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TABLE 26 



Effective Price of Reading DoD Technical Reports by 
DoD-Punded Scientists and Engmeers: 1982 



Cost Category 



Report Purchase 

Processing 

Identification 
TAB 
GAB 
Search 

Other Announce. Pubs. 
Other 

Access 

Reading 

TOTAL 



Average 

Per 
Reading 

($) 

$ 0.10 



3.30 



12.30 
7.30 
5.90 

30.70 
0 

5.60 
28.60 
$ 4A.80 



NuE±>er of 
Readings 
( Millions) 

12.4 



12.4 



1.5 
0.7 
6.0 
1.0 
3.2 

12.4 

12.4 

12.4 



Total 
Value 
to Readers 
( $ Millions) 

$ 1.2 



40.9 

18.5 
5.1 
35.4 
30.7 

0 

69.4 
354.6 
$555.8 



Source: King Research, Inc. 
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his representative) for searching and reading. In the case of TAB the 
total expenditures is $74.9 million (i.e., 1.5 million readings from 
searches times $49.90 per reading). If a substitute methorJ of searching 
costs more then, in order to mintain the $74.9 million fixed expenditures, 
fewer readings would be possible. In this instance there would be about 
one million readings instead of 1.5 million readings from TAB searches. 
Ihe value lost in terms of the scientist's or engineer's willingness to pay 
would be about $25 million . Similarly , the total amount of CAB 
expenditures is $31.4 million and the nuirber of searches possible at $75.40 
would be about 400 thousand instead of 700 tho^^sand. Ihus, the value lost 
in this instance would be about $13 million^ Finally, the total value of 
bibliographic searching by computer is $261 million and the number of 
searches possible using substitutes is 5.1 million instead of 6 millioi so 
that the value lost in this instance is about $39 million. Ihe total 
amount of value lost (in terms of willingness to pay) comes to about $77 
million for the DTIC bibliographic products and services (TAB, CAB and 
conputer searches) . These numbers are all considered to be lower bounds on 
the amount of value achieved through use of the OTIC bibliographic products 
and Services . 

4.2 Value in Terms of Savings 

In the survey, respondents were asked to indicate the amount of 
savings that were achieved by then as a direct result of having read a EtoD 
technical report (i.e., their most recent reading of a t^hnical report. 
These savings can be considered a measure of the value of information to 
the funders. The average such savings stated by the survey respondents was 
$4,700 per reading; where about 85 percent of the readings of technical 
reports yielded some savings. These savings involved label and equipment 
as typified in the following statements: 



52 



XAB1£ 27 



Value of TABj CAB and Ccraputer Searchfis in Terms of Willingness to 
Pay by DoD -Funded Scientists and Engineers: 1982 

($ Millions) 



Total Readings 

Average Value in Will- 
ingness to Pay 

Valxje Directly Attribu- 
table to Product or 
Service 

Readings Lost by Sub- 
stituting 

Value Found by Substi- 
tuting for Product 
or Service 



TAB GAB 

1,5 million 0,7 mllion 

$ 49,90 $ 44,90 

$74,9 million $31,4 million 

500,000 300,000 

$25 million $13 million 



Computer 

Searching Total 

6,0 million 8.2 million 

$ 43,50 $ 45,70 

$261 million $367 million 

900,000 1,700,000 

$39 million $77 million 



Source: King Research, Inc, 
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o 



OTIC provided some needed infonnation on who was involved 
in a prograin and gave foreign contacts. 



o Because of the nature of the infonnation^ some (feta can be 

found only through DTIC and therefore is priceless. 

o The infoniBtion gleaned from the search gave insight into 

something the researcher had not thought of before. 
Provided insight into the problem. 

o Gave the contractor an idea of what Department of Defense 

was interested in. 

o Saved in writing and research time since someone else had 

tested the infonnation already. 

All such responses are listed in Appendix A of the report, it is noted 
that results observed with Department of Energy sci^tists and engineers 
were som^rtiat lower on the average. DOE scientists and engineers indicated 
an average savings of about $1,280. They reported savings for about 75 
percent of the readings of technical reports. 

The total value to funders, measured in terms of savings achieved 
through the 12.4 milliCHi readings of DoD technical reports, is estimated to 
be about $58 billi<»i. The amount of this attributable to technical reports 
read as a result of using TAB, CAB and bibliographic corf)uter searches is 
eUx)ut $37.5 billion. The value of savings to funders found by substituting 
bibliographic products and services for TAB, CAB and bibliographic con^xater 
services is estiirated to be eUx)ut $8.3 billion. These <feta are surtnarized 
in T^le 28. Like all of the estimates of value derived in this Chapter, 
these figures represent value realized in a particular year (1982) from use 
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TABLE 28 



Value of TAB, GAB and Computer Searches in Terms of Savings to binders by 
DoD-Punded Scientists and Engineers : 1982 



Computer 

TAB GAB Searching Total 



Total Readings 1.5 million 0.7 million 6.0 million 8.2 million 

Average Value in Savings $ 5.600 $ 5,500 $ 4,200 $ 4,570 

Value Attributable to 

Product or Service $ 8.4 billion $ 3.9 billion $25.2 billion $37.5 billion 

Readings lost by Sub- 
stitution 500 .000 300 , 000 900 . 000 1 . 700 . 000 

Value Found by Substi- 
tuting for Produce 

or Service $ 2.8 billion $ 1.7 billion $ 3.8 billion $8.3 billion 



Source: King Research. Inc. 
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of the total OTIC data base, that is, the cuinulation of the reports and 
bibliographic records processed over the years. Ttius the investment 
necessary to generate these levels of value includes both 1982 expenditures 
and earlier expenditures on collecting and processing the DTIC collection. 

The Defense Technical Information Center also provides infontBtion 
froni management data banks as Toentioned in Sections 2 and 3. Relative to 
the corrpleteness and usefulness of the data in the data banks, users were 
asked to place a value on staff time and equipment and supply costs saved 
by using the OTIC nanagement databanks as shown in Table 29. Eighty-cne 
percent of WUIS users found they saved money by using the databanks showing 
an average of $2,600 per user per search. Eighty-eight percent of R&DPP 
users saved an average of $3,600 per user per search and finally, 74 per- 
cent of the IRfitD users saved $3,900 cn the average. 

Overall, 9 percent of the survey respondents used iranagement infor- 
mation reports. It is estiireted, therefore, that there are approxiiretely 
14,000 uses in total at an average savings of $2,900 per search. The total 
yearly savings resulting from nanagement data bank use is thus $73 million. 

4.3 Value AssQCi;^l-prl with Levels of Document Classification 

To this point, the value associated with technical reports has been 
discussed for all DoD r^x:>rts, regardless of their classification level 
(classified, unclassified/limited, or unclassified/unlimited) . An im- 
portant issue, however, is the extent that classification has on extent of 
use and hence on value. 

In Section 3 it was indicated that 3,169 titles ^ ^ 1982 were 
classified, 8,048 were unclassified but limited and 16,089 were unclassi- 
fied and unlimited. However, the nuirt:>er of readings in 1982 of classified 
technical reports was 2.6 million readings conpared to 2.6 million for 
unclassif ie<Vliniited and 7.2 million unclassified and unlimited. This 
means that the average nun±)er of readings per title (recently announced) is 
about twice as high for classified reports (820) as unclassif iei^limited 
(323) and unclassif iei^unlimited (448). Even though the average number of 
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TABLE 29 

Average Value in Terms of Savings of DTIC Managenient Data Bank Uses: 1982 



Proportion 

With Savings Average 

Data Bank C/j $ Savings Value 

RST Work Unit 81 $2,600 

R£tD Program Planning 88 3,600 

Independent R&D 74 3,900 

Total 82 $2,900 



Sjurce: King Research, Inc. 
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readings per title may be higher because rrany of the 2.6 million readings 
of classified technical reports were of older titles (i.e. , older than 
1982), the data do suggest that the amount of reading of classified r^)orts 
is greater than that of unclassified ones. ThuSf the value of classified 
reports in terms of willingness to pay and in savLngs is also likely to be 
greater. In fact, it is found that the average value in terms of savings 
is substantially higher for reading classified technical reports than 
unclassified/limited or unclassified/unlimited reports as shown in l^ble 



A further, and perhaps^ even more iirportant question is ho*;; much 
v.!i]ue is lost because of the constraints imposed on classified technical 
report distribution and resultant loss of readings. This question cannot 
be answered directly, although some evidence can be shown. It is found 
that nearly all readings of classified technical reports cocne from reports 
that are identified by OTIC bibliographic products or services, whereas 
about one-third of the readings of unclassified and unlimited technical 
reports are identified by other means. This suggests that an additional 
(at least) one^third readings would be made of classified technical reports 
if there was greater awareness of than. If this assumption is correct, 
there would be about one million additional readings of classified technic 
cal reports at an assumed average value in terms of savings of $8^600. This 
comes to an additional $8.6 billion in value attributable to classified 
technical reports or $5.1 billion if the readings of classified technical 
reports replaced the readings of unclassified and unlimited technical 
reports. This analysis, while based on a nunt)er of assurrptionsf does 
support substantial value may be lost by restricted announcement and dis^ 
tribution of classified technical reports. Phrased in another way, poten- 
tial losses in reading and value are costs which must be balanced against 
the other k inds of benefits which accrue from classification of DoD 
reports. 
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TABLE 30 



Average Valxje in Terms of Savings of DoD Technical Report Readings by 
level of Report Classification: 1982 



Report level of 
Classification 

Classified 

Uhclassif ied/ limited 

Unci ass i f ied/ xjnlimi ted 



Average Savings 
Valtie 

$8,600 

2,400 

3>500 



Total 



$4,700 



Source: Defense Technical Information Center and King Research, Inc. Survey of 
DTIC Use, 1982. 
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Answers to Open-Ehded Oaestions 



Question: •Wfere the results useful for that purpose? 
Wiy?" (See page C-2.) 

Some infonretion 2-3 years old gave some contacts in field. 
Provided one stucJy on tC5)ics and added to other informatioi obtained, 
(^ve some info he wasn't knowledgeable of, 
(^ve info contributing to pallets, 

This organizations needs to know about work in progress and work corrpleted 

to avoid duplications of work with other organizations. 
Picked out individual sites for possible audit. 

Did not show any work being done in area. Project got funded — got info 

wouldn't otherwise have. 
I did find relevant documents. 

To avoid duplication of effort. To save travel time and money and to help 

defend the NASA budget. 
It helped me avoid duplication of work effort made by others. It helped me 

write a Military Standard on Logistics Models. 
It helped me avoid duplication of effort with other organizations engaged 

in similar work . 
Einabled sea rch to obtain documents on the subject. 
Gave me the needed info to begin Law program. 
Several documents were pertinent. 
Received info to pass on to others. 

(^ve info on Ceramics for high tenperature and everyone was happy about the 
report . 

Report abreast of present technology. 
Hade manager more aware of relevant data. 
Hjch of info was several years old. 

Did indicate a certain trend, but the data was truncated and did not give 

the latest figures. 
Material was current. 

Report useful to this inmediate group and an engineering group. 
Received needed info for proposal. 

(^ve update of methods and dealt with problems on subject, 

gave needed info as to where to find other info. 
Some items didn't show upi but was partially useful; didn't use all key 

words. 

Now I only have one piece of paper instead of multiple copies or pieces. 
Accessions were applicable to our Project. 
Called author and got report# which is useful. 

Helped to identify the ongoing work and other agencies also working on 
same subject. 

Negative/ found nothing. 

Wbrk with vague out of date and inaccurate not complete. 

Except time it takes to obtain follow on reports from DTIC. Can call 

author . 
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Vhat you do-assessment of research-apply to Public Law 9648. 
Wrote an lOM on subject as a result of the search. 
Other obligation presented. 

OJestion of no info. Perhaps definition of terms was inconplete. 
Usefulf but inuch he repeated with paraineter search. 
Gbve broad based backgrounds speeds up research. 

Gave significant literature background for in-house development and 

publication. 
Nothing on behavioral. 

Provides info that probably wouldn't otherwise. Have to assume studied 

program is coirplete and consider all available info research. 
Gbve broad index for this field. 

Sound relevent info-obtain leads for use in project. 

We were getting ready to do our own searching and did not have to. 

No info found . 

Is it a real program? Some deemed essential/ some non-esseitial-like to 

match what they can do with what services need. 
Provided insight into problem. 

Some programs indicate we can modify equipment to neke it of interest to 
the 

con^ny . 
Provided the info I needed. 
No info in data base. 

Especially for future purpose . Info for l^St_ system minimal. Very 

helpful for next system. 
Provided what others were doing^who was active in field. 
Applicable but request was cancelled. 
Very helpful-DTIC did a good job-received a lot of info. 
Gbve info to group^iowever some info old. 
Fulfilled req. -didn't locate info., so let contract. 
Gbve several studies done in field. 
Found info on what others were doing. 

Said what other people were doing and gave foreign contacts; but wanted 

more up-to--(fete info. 
Not up-to-date. 

Provided source info that could be used and contacted. 
Sole service available. 

Gbve extensive info on how to reduce costs. 

Gbve status of items being worked on in labs and confirmed what already 

knew and gave additioial insight. 
Provided needed background info. 

Able to find some areas on interest and where funding was. 
Large listing of info. 

Found info useful in finding out what Defense was interested in. 
Gbve info needed. 

Identified other ccMtpanies/agencies who are doing research. 
Very satisfied with DTIC personnel-but didn*t get all info needed- 
Keywords are a probl^. 



A-2 



72 



Need more advar*ce info^however report confirmed this info wasn 't 
available. 

Good source for current research-is cojrbined RDPP-WUIS search. 
Got othervise naifindable soource of info. 

Provide useful info in fulfilling requirements of cot tracts-provide 
excellent 

background info. 
TSiey were applicable to the info. I needed. 
G^ve up-to-date info. 

G^ve more tnan needed- but got info needed. 
Confirmed what cocrpany was going to do- very supportive. 
Because there is very little studies available on Lantim. 
Alot of background from others' research. 

Knowle<^e of related industry studies and gives leads to contact. 
Because it outlined work done by others and they used it to analyze their 
program. 

G^ve several company contacts and info. 

Several relevant citations were included in results. 

We are building a new theoretical model and several searches were received 

and af^licable. 
TSiere were several pertinent references-^s quite helpful. 
Yes, the results were very useful and several documents were relevant. 
I was able to use the info I received, 
teterial included was indeed updated info. 
We are working on a project and needed all avail, info. 

Dave needed info on what was involved in program- gave contacts-described 

program. 
Assisted in study. 

G^ve specific info as to the contact and budget info. 
It detailed work being done. 
Insight into something they had not thought of. 
Too broad scope. 

Received sufficient background data for her report. 
Some of what he was doing had been done before. 

G^ve necessary history of present torpedo work and appreciation for v^at is 
^oing on. Don't know-hasn't finished reading and collecting data. 

Found references I would not have found otherwise-other contacts were made 
available to me. 

I got some info that was useful. 

A lot of leads- but no documents that were ordered ever came. 
Data seemed insufficient fot the search in questiai^ta was used in other 
efforts. 

TSiere were some relevant cites in the document. 
"I did not receive any pertinent info from OTIC." 
Pertinent Info. 

Helped link other info concerning state of the art in Millimeter Vfave 

Technology . 
Any bit of info is useful. 
Several pertinent searches were received. 
Small amount available. 
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Pertinent References. 

Found reports he hopes will be useful. Has not had time to fully studh^. 

Because it was pertinent to iny requirements. 

He was disappointed in the sroall amount of material. 

Helpful in providing what info was available. 

We found that little was being done in the subject-need to do more. 
Identified number of similar R&D efforts- 

showed work not being done elsewhere. 
<^ve knowledge of other power sources for anny stud^^^. 
G^^^e needed background info. 



Question; "Could you describe how it saved you time?" 

Wbuld have had to perform search on own. 

Would have had to get library to search other sources. 

Wbuld have had to get library to search other sources. 

Would have had to perform search in other areas-i .e. library, other 
systems* 

Manual search - UCLA had ^tl7 saves some time; 1/2 hour versus days time. 

Wbuld have had to perform library searches & contact personnel to gather 
info. Only used report as back-up & no significant savings resulted - 
probably wouldn't have searched info if not available through OTIC TRs. 

Had to repeat research with all its mistakes to find the right answer. 

Very difficult to find their type of work through any other source. 

They haven't saved time but has added to his research effort. 

Also searches STAR, stlil has to find out material from other sources. Gets 

a lot of data by word of mouth . 

Saved the^m from constructing research that was done before. 

Wbuld have to search for info elsewhere & some info not available anywhere 

besides OTIC. 
Wbuld have to search info elsewhere. 
Had to spend time looking elsewhere. 

We would have to spend time travelling to other areas plus searching for 
the info. 

Believes us from lots of searching elsewdiere. 
teny hours of hand searching. 
Searching through the library. 
Kany hours digging manually. 
Time searching manually. 

Wbuld have had to make a thorough literature search. 
Don't know. 

Needs material within 1 to 2 days-Most bnc materials arrive too late to be 
of use except in confirming their findings to a problem. 
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Able to respond quickly to corporate headquarters in LaJolla, California, 
tenually searches indexes in open literature. Allows good background 

material which they would not have had time for otherwise. 
Saved literature searching time. 

Able to respond faster to needs of our research group. 
Tum-around time is faster using OTIC. 

teke decision of info needed, then go out to recreate info, after search 

was made for other sources. 
Saved time by not having to recreate data. 

This was ordered for a potential contract. If they get the contract for 
developing a product, they could realize savings of $100,000 tovrard that 
develc^^inent . 

Put subject in direct contact with needed info-settle a technical issue. 
No savings conpared to the terminals but gives me time to browse & gave a 

broader outlook, 
rbterial could not be found elsewhere so fast. 

Saved literature searching time when DFOL5 conputer terminal was down. 
Turnaround time is fasdter using CTIC. 

Results are more corrplete and data is more quickly accessed than elsewhere. 

Saved literature searching time. 

TAB used as baukup when DFOL5 corrputer down. 

Turnaround time is faster using CTIC service. Provided info that we didn't 
currently have. 

Finding reports in TAB saves time^ Reduces the number of places to look. 

Over the course of a project, the documents involved can save $15,000 on 

$3,500,000 project. 
G^ve additional references to P Info. 

Did not need to order these reports. Already knew data contained in these 

reports. Normally it does give new info. 
By not having to spend time doing library research, etc. 
In doing my own travelling and searching. 
Researching through library. 

Not having to repeat the research that was done. 
Planning time and time in develC5)ing procedures. 
Did not have to leave work area for research. 
Research time. 

Discovered that the work had not been done. Beginning of a new project. 
It was negative inforTTBtior\/still alot of work to be done on this project. 
We were able to get the infonretion. 

Not a technical report in specific, but a assessment of technology-not 

possible to put a dollar value on. 
Sole source of information-only place to get it. 
Went to other documents for the particular information. 
TAB is only source of information. 

Did not have to repeat research. Had a reliable source of informatioi. 
It gives one a better view of the problem. Documents found quickly. 
Don't have to go through all journals. Don't have to go through ccitpanies. 
Time spent going elsewhere, and checking with other colleagues. 
Did not have to duplicate research. Data was there. 
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Don't have to do mechanical search, but at this time it does not seem that 

survey is coinplete. Needs to change search strategy. 
Not address the subject. 

Would have had to go to library and search for informaticr 
Would have had to perform own research. 



One centralized source for much information. 
Two weeks of corrputational work. 
More research needed. 
Would have had to "Re-invent the wheel". 
I got it proirptly when I needed it most. 
The entire system was down and I needed a quick response. 
I did not have to look through all literature sources to find what I 
wanted. 

We might have lost out on an RFP of we hadn't gotten the information 
pronptly. 

The coiTputerized on-line system was not operating and I needed to reach 
pronptly. 

We did not duplicate listing and research which had already been done. 

It saved my having to search all the literature. 

Reduced research time, freeing me for other tasks. 

Had to have done research himself. 

tfeke more phone calls. 

Time spent not having to do research. 

Two-Three days of work saved not having to do the search elsewhere. 

Would have had a lower quality product because he would no have had this 
info . Since most of what he uses is classified, would have been 
difficult, if not irzpossible, to get info elsewhere. By having OTIC's 
reports available, didn't have to go elsewhere to find info. 

Didn't have to go elsevrfiere to find info. 

Library would have to search. 

Know about info that wouldn't have known about otherwise- nay not have had 

the info available. 
Would have had to contact individuals in this area and have discussions 

with them. 

Time and travel to do research by other nethods. 
Telephone interviews and traveling. 

Would have had to look for other bibs, and libraries and ^end time doing 
personal research. 

Vfes able to see methodology that could be applied- didn't waste time using 
methodology that didn't work. 

Time saved in having terminal instead of going to OTIC with key words. 
Don't know yet how reading reports saved time* 
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Would not have had most of info-has not been successful in locating 

materials in past. 
Would have called agencies if knew who to contact-crriC is irreplaceable 
crriC is necessary to his work-could not be duplicated therefore. 
Travel time for moving pice to place & time for searching. 
Searching . 

We were relieved of doing our own searching. 

We would have had to do our wn searches. 

Our time was saved by someone else doing the looking. 

Time spent s^rching. 

Staff did not have to devote time to searching., 

Not having to do our own searching. 

We would have spent several hours doing searches. 

Not having to do our own research for revelant subject natter. 

Travel & not having to do our own searching through volumes of other 

neterial. 
Doing research. 

We would have had 3 or 4 other people working, searching for these reports. 
By not having to do our cwn searching. 

TSie last report one read was not applicable, however when they are, I save 

time by not having to look for info elsev^ere. 
Would have had to look through other references-textbooks other info. 
Would have had to write a conputer program to obtain the analysis after 

digging up the material. 
Finds work already done and people to ontact. 

Relevancy 90% vs. 0% - 1%. Keep up a leading edge of state-ofH:he-art. 

Would have called government agencies.. 

I read about it and was able to identify it prorrptly. 

Vtould have developed it or gc^e without-probably the latter. 

Vtould have researched other sources such as KTIS, chenical societies. 

Would have used other data bases or nenual search of literature. 

(Hasn't received items yet). 

Would have had to do alot of basic work, with uncertainty and risk in 

securing findings. 
Added to the base of knowledge for this project. 
Speeds up tinte in finding materials. 

We were able to respond prcnptly to a DOD request for prc^sal. 
It saved search etime response becasue our computer terminal was not 
operational. 

We were able to obtain contracting and subcontracting leads. 
Ihe data was not readily availabl eelsewhere. 

Informtim was readily available to take advantage of a DOD procurement 

invitation 
Saved searching time. 
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It is not readily available elsewhere. We could not operate so well 
without OTIC products Keep up with the state-of^he^art and can work 
more effectively. 

Would have contacted people in field for reports. 

Would have done own search or performed library search. 

No other way to get the infonration. 

A void in this particular area saved time and transit. 

It helped us react pronptly in preparation of a research proposal. 

We would have to duplicate test results and we were able to decide that we 
also wanted the classified reports. 

Saved time having to look elsewhere. 

Would have had library search frcsn other sources. 

Library would have done searches frOTi other sources. 

Would have coirpleted job on own knowledge/experience and perhaps go to 
experts in field. 

Would have conpleted on own knowledge/experinece aknd perhaps go to experts 

in field. 
Would not have gotten the info. 

Would have gone to library to secure some of the info. 

Would have qone to the library to secure sane of the info. 

Would have used other indexes- would have taken the same airount of time 

that OTIC does, however he prefers our service because it gives him more 

info. 

Would have used other indexes- would have the same amount of time that OTIC 

does; however, OTIC gives more info. 
Would not have had access to info. 
Search time. 

We did not have to spend our time searching. 

By having OTIC identify relevant abstracts. 

The searching for relevant documents. 

By not having to do our own searches. 

Telephone calls, direct contact, etc. 

Trips, tel^hone calls, etc. 

Time saved in getting document easily. 

Look at CAB at getting something extra; not as a savings- would not g^ 

material otherwise. 
Did not have to go to many other soruces to get the same info. 
The CAB saves the manual work offuinding the docum^ts, may of hwichcould 

be missed. 
Need to find info, elsewhere. 

This research is done as if a personal SDI-is the only way he searches^ 
Does not have idea of how long it would take to get this info in other 
nanners. 

Collection of (feta that would have had to have been gathered from may other 
sources. 

Do not have to write to research groups to get the info. 
Need to have gone back to the source stuc3y. 
Getting material eliminates reinventing the wheel. 
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Having automated searching speeds up finding the documents. 
Wbuld have gone to organization who issued r^>ort. 
Wbuld have generated info hi^nself . 

Wbuld have canvassed other services for info./reports. 

Wbuld have used other databases/reports-he uses broad resources. 

Wbuld have contacted individual pec5)le. 

Wbuld have used other search methods in library 

Survey type report-lot of research bibs-did alot of initial sorting. 
Done for a special project. 

Have to get data from other sources-report quite conprehensive. 

Documaits used to conplete fact sheet on data needed in 24 hours. Would 

have taken much longer to find data wanted on short notice. 
Rave access to info so research was not repeated. 
Showed that the work had not been done. 
Soneone else has develc^>ed a concept. 
Time saved in reduction of searching. 
Hake us aware of current trends. 

WDuld have contacted Federal agencies and contractors-but wouldn't have 
equivalent info. 

Wbuld have called Federal agancies ancVor contacted contractors. 
Info was delivered promptly by library so that briefing was prepared on 
time. 

teterial was not available elsewhere and were very specific. 
Titles were available promptly for my briefing. 

Material was probably available. Saves time designing instructional 
materials. 

We did not have to provide secondary distribution of our own handbook. 
Saved search time becasue DRDLS computer was down. 
Brought us up to date on current concepts. 
Background material. 

Time saving in searching through TAB-<:ritical for doing research-^ However, 
TAB is not very conplete on the Behavioral Scieinces. Feeling that it 
is an excellent service. 

Wbuld have made personal contacts in his field of interest. 

Wbuld have gone to author to ^ccure report. 

Wbuld have gotten a contractor to secure info. 

Wbuld have used private databases or gone to universities. 



A-9 

ERLC 



Wbuld have performed studies personaly. 
Hasn't received reports. 

WDuld have contacted authors after lengthy search for report titles. 
Did not have literature searches. 

Time saved by getting reports- No idea of how long it would take to get 

rQX)rts otherwise. 
Do method of evaluating time saved. Still in training. 
Turnaround time is faster using UTIC products. 
Saved time because DRDLS corputer terminal was not operating. 
We were able to respond promptly to the researchers request. 
Turnaround time is faster using OTIC products and services. 
We are six to eight weeks ahead of schedule using CAB. 
Vfe could not have found it elsewere. 
Saved time because the computer terminal was down. 
Turnaround time is faster using UTIC services. 
Turnaround time is faster for us using t3TIC. 
Turnaround tijne is faster for us using DTIC. 

Wbuld have' reinvented the wheel** -contacted people in field of interest. 
Reduced time involved-enabled to get info quickly. 
Narrowing down the search- Having to use other sources. 
Gives outside contacts for further info. 

Wbuld have called other activities involved in some area of interest to get 
reports. 

Wbuld have called agencies for info. 

Wbuld have searched for people in the field to speak with. 

Wbuld have to research different Agencies for information- calf travel. 

No response. 

Expect report to save time in giving much background informtiai for future 
re sea rch. 

A case of looking for informtion. Information is not available through 

other suorces. 
Wbuld have searched in *'open*' literature. 

Wbuld have contracted other sources such as people he knows in the field. 
Wbuld have been ""lost** without the services-significant time to secure info 

from other sources if ever possible. 
Wbuld have been ""lost** without the services-time to secure info from other 

sources if ever possible. 
Would have used info- not that critical to work. 
Wbuld have gone to agency involved to secure info/report. 
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Would have calleci other agencies for reports or performed research. 
Conclusion led theni to drop filers as a program. This could lead to 

enormous savings. 
Don't have to do a alot of running about to find reports. 
Classroom reading-vill probably add time of the student. 
Better defined a research tc5)ic. 

CAB saves time in not having to track down these documents (citatoins) each 
week. 

Additional research material that was essential to task. Did not have to 

recreate data of report. 
Would have contracted out to acxjuire info. 
Wouldn^t have used that info in projects. 

Would have gone to report source to acquire info (if known) otherwise would 
do without . 

l*>uld have contacted government agencies of people in field of interest. 
Would have contacted government agenies for info/reports. 
Would have contacted a local university. 
Time saved in method of accessing report. 

The reports are the job- Quickness of access saves time-2 days work saved 
in not having to walk through bib. material (Savings at 1^ Lt at 2-4 
year salary) . 

Would have conducted interviews. 

Report is stlil to be read- will require alot of time. 

Would take up to a week of time to find the reports which the CAB gives 
him. 

Saved him from having to perform the same work. 

Til is is saving given is on search time saved by finding bibligraphic 
online. 

Saved the drudgery of a difficult iranual search. 

Would have not been aware of the nifo- would have performed own search. 
This is his work. 

Depends on savings of preventing the need for future research. 

we were able ot respond promptly to our research staff needs. 
Tliis type of intelligence was not readily available elsewhere. 
Travel time to do searching- also spending time on manual searching. 
Identifying from the abstracts available. 
^fenual searching . 
OTIC did the work. 

Applied to future planning and professional awareness. 

Ehables him to get needed info in a short period of time. Ehables him to 

be more productive- get more info. 
Would have to do iranual- nanual search probably would have taken a month. 
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Would not be accessible without oi-line database set up the way it is. 
May have saved someone down the line time. 
Did not have to do manual search. 
Did not have to duplicate research. 

Not having to duplicate research. Time saved in identifying additioial 

references in each report. 
By not having to search elsewhere. 
Time spent on researching. 
Didn't have to find the info elsewhere. 
Having OTIC do the research for us. 
By having the research done for us. 

By having OTIC ooing the researching, and travel time also. 

We are all spoiled by using the computer- do not have to research in 

libra ry . 
Travel to get reports. 
Gain info- mking a point. 

Haven 't gotten data yet, so can 't evaluate. Data in past ht been 
fruitful. 

This is for a gain in material, not a savijigs of time. Work provided info. 
All work done by staff and aides. He does not pay any attention to it. 
Did not have to recreate processes of establishing the methods of 

resea rch . 
Info readily available. 

Did not have to repeat research^ coulo be from 2 to 10 mn-years of work. 
Only place where the research is available. Can't make estimate about the 

time it takes to get these things, 
Oaicker to do, much more accurate, 
COntributioi to project. 
Didri't have to visit the source. 

Given much better background on U.S, capabilities. How to direct efforts 

toward foreign technologies. 
Read to broaden knowledge. 
Can find out what is done. 

Otherwise have to do less productive and more time consuming 

research-mainly interested in military afi^lication. 
Try to set up definition themselves, then get it approved. 
V*tere to find available info. 
Add value to the project. 

Did not have to spend time that a manual search would have taken. 
Did not have to recreate research. 

Design our survey and acquire results to use on projects. 

Some of the info would not be directly available and would have to contact 

other organizations for info. 
Would have had to call other organizaitois for infonration/materials. 
Research field of study & probably result in duplicate effort. Wbuldn't 
have the opportunity to build on someone else's work. 
Would have had to go to the Records center to search records for answer. 
Have to go to library and research and review articles. 
Call originating agency- how to obtain. 

Would have a lower level of technology and be ignorant of info in this- 
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would ^ave had to perfom our research but would not have obtained total 
info. 

Go to specialist at Lockheed-This would have reuqired a trip. 

Itens to be found at one spot. 

Not a savings of time but a gain of infonnation. 

This was an initial search to see if info on abstracts would show problems 
of finding things, as current key words did not match the aproved DRIT 
list . 

In conjunction with another report could respond to a Logist ics need 
Sunmary withajt survey of 20 other reports. 
Did not have to contract search out. 

Value given to the providing of info. Material not known to exist 
elsewhere. 

Without research aids such as this, valuable data would be missed. 
Haven't done reading. Can save time only if another has soinething new to 
offer. 

Conputer does the work. Need to repeat research and look at a lot of other 
places. 

Did not have to go elsewhere- data easily provided, teke more efficient 

use of time. 
Had a read access to the material. 

Iten used to keep aware of current state of research in the field. 
Still examining materials to begin project. 
Did not have to reconstruct a war game. 
TAB is a source- without it he would not know work exists. 
Although more work has resulted, time was saved by not having to search for 
materials. 

Would have had to go to Washington, D.C. and perform search and would take 
2-3 days. 

Vtould have had to go to library and perform search of materials. 
Not read yet. 

Wbuld have had to duplicate most of repots on research library for ohter 

sources of info. 
Not having to do the research. 
Do our own research. 

Reading was a gain of data; used to find other reports from bibiliography. 
Did not have to make trips to get documents. Bibliographies there to check 

off against current holdings. 
Would have t^ go to other source to get infonnation through cojTfnercial 

access- more time consuming, more expensive. 
Did not have to duplicate research. 
TAB acts as a good source of info. 

Wbulcii't have gotten all the info but would have searched in library. 
Spend time in library searching for infonnation. 
Go to library or other suorces to find info. 
Go to other sources for info. 
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There is a much greater speed of researching on our data team. 
Would have used ERIC data base. 

Provided extra info for the project on hand. Did not have to recreate a 
sea rch . 

Documoits found at a single loccition. 

Wouldn't have known about reports- because they're limited- probably 
wouldn't have info- but would do some library research. 

It gives him a coivenient, quick summary of what is current, but he insists 
it does not save him time or nmoney. 

Saved literature searching time. 

teterial could not be found elsewhere. 

Saved literature searching time. 

Spend the hours of basic research plus put it together. 
Short time to get classified info. 

There were delays in the initial ordering process. Report enabled this 
research to be up to date and not repeat research. Can also learn 
successful techniques. 

toy have saved 1 say-possibly 6 weeks time. 



Question: "What did you and your staff save as a result of that 
reading? (See pages C-7, C-8.) 



Readily available information. 
Time from looking for other sources 
Time 

Spend time learning about answers to particular problons. 
Don't know six months of trial and error. 

Not yet arrived- will save time if they live up to their abstract 

Did't have to do the work described in report. 

Time. 

Time. 

Time. 

Time/Money. 

Time. 

Time. 

Time. 

Time. 

Time. 

Tim^ and ^^oney. 

Time and Honey. 

Time/Money 

Time/Money. 

4 days of Effort. 

2 days on each. 
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Answered a question. 
Time/Mon^. 
Time and Honey. 
Time and Honey. 
Locating infontation. 

"Why do you ask these questions^ it is an obvious consideration that any 
very recent search would not yet have produced docunvents. Also are we 
placing value on the BIB or on the documents themselves." 

Got more info. 

Time. 

0. 

Time/Money. 
Time. 

Time/Kon^. 
Time/Honey. 

Planning time and Time in Deveoping procedures. 
Time and Honey. 



Nothing. 



Used in preparing Handbook. 
Time. 

Saved "Re-inventing the wheel". 
Time saved. 

Time. 

Time- did not have to "re-invent the wheel". 
Time 6t Money. 
One hour . 

^bnhours and Money. 
One hour . 
Time and money. 
$30. 

Ctorpleteiess. 
Hours of time. 

Got results from what others have already done. 

By having DTlC's reports available, didn't have to go elsewhere to find 
info. 

Didn*t have to go elsewhere to find info. 
Library would have to search. 

G&ined access to sonething they didn't know about. 

Would have had to contact individuals in this area and have discussions 

with then. 
Time. 
Time. 
40 hours. 



Saved pecsoial time in searching info in library. 

Vfes able to see methodology that could be applied^ didn't waste time using 

methodology that didn't work. 
Could not get info otherwise. 
Time. 
Time. 

Time and Honey. 
Time and Hoiey. 
Time. 
Time. 

Time and Money. 
Time and Money. 
Time and Money. 
Time anc3 Honey. 
Time and Money. 

Could do other things; one week of searching elsewhere. 

Plagarize Analysis. 

Vtork alrea*; done- have to meet. 

Time. 

Money//Man;.ours Time/Cost. 

Time. 

Time. 

Better quality reports. 



Time and M(xiey. 
Time and Money. 
Time and Moiey. 
Time and Money. 
Time and Money. 
Time. 
Time. 

Invaluable — no other source. 

Time and Money. 
Time and Money. 
No answer . 
Time and Money 
Time and Honey. 

Confirmed prior knowledge. 

Time. 

Time. 

Time* 

Time. 

Time. 

Time. 

Time and Money. 

Time. 

Time. 
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Time. 
Time. 

Personal Time. 

TiTT>e/Cost . 

Time/Cost. 

Saves patrons time. 

Cost. 

Time. 

Cost" Too lengthy time. 
G^ve answer. 
Time and Money. 
Time. 

Got info probably woulcii't have acess to. 

Time and Money. 

Time and Honey. 

Time and Money. 

Time. 

Time. 

Time. 

Time. 

Time. 

Time/cost 

Time. 

"This is the same result as for Part I" (no result given in part I.) 

Time and Money. 

Time and Money. 

Time and Money. 

Time and Money. 

Time and Money. 

Time and Money. 

Time and Money. 

Ctost. 

Cost. 

Reducecl time involved^ Enabled to get info faster. 

Time. 

Time. 

Time. 

Time. 

Time. 

Time/Costs. 



Time. 
Time. 
Cost. 

Personal interest. 

Time. 

Time. 

Cost and time to do our own research. 
Ctjst and time to do our cwn research. 
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Using TAB saves time in finding documents. 
CostAinie. 

Had info readily available. 
Better effectiveness. 
Better effectiveness. 
Time. 

Time. 
Time. 
Time. 

Time/Cost . 
Cost . 

Time and ^5oney. 
Time. 



Time and Honey. 
Time. 

Article Screened ahead ofTime. 
Tremendous amount of Time. 
Time. 

Develops development of corrputer models. 

Saved tiine. 

Time. 

Time . 

Time, 

Reliabiltiy and credibility. 
Got graphs of VS technologies. 

Ehdless hours of figuring things out when the material is available in 

literature 2 years of work. 
Not available if not found here. 
Better job with info. 
Good info. 

Familiar with areas of interest. 

Made info available for instructional purposes. 

Design our survey and acquire results ot use on projects. 

Time Duplicate effort. 

Several months. 

Time. 

2 to about 20,000. 
Time. 



In forma tioi . 
0. 

5% of effort that would be expended elsev^ere. 
Presented a good simulation- saved several months. 

Although more work has resulted^ time was saved by not having to search for 
materials. 

Acticfis/projects have resulted frwi reading report. 

Reports were used for browsing only and savings haven't been realized yet. 
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Time. 

^^ot read yet . 
Time. 

Gives info. 

CXits dcwn or second guessing-better factual basis for raking decisions. 
Time. 

G^ve info to complete projects-saved time. 
Time. 

Made organization more eff icieint-time saved. 
"Time. 

Avoided olicate effort and used their recortmendations. 
Time/Mc: ^- . 
Time/Honey. 
Research time. 

Indispensable, essential access in short time work being done. 



Question; We are interested in your use of reports provided by EHTIC. Can 
you tell us what report you read last? Title; (See C-6.) 

Anaylsis of future develc^ent witn in Air Force. 

Plasticity. 

High speed bearings 

Don't Know. 

G^s Chromotography. 

Active Aperture. 

Developmeit and Utilization of Integrated and Malti-dim^sions salary. 
Group on forecasting methods for defense contractors. 
Infrared Detection Unit. 

Surveillance; Navy Fleet, Fire Exiting. Cartridge. 

Mechanical Properties of 70/50 T-73. 

Fracture Toughness Data for Disc. 

Nuclear Fuel Reprocessing. 

tedar Signal Processor. 

Vfersaw Pact Logistics Planning Guide. 

Develcnnent of a Low Cost EMP Protection for Ops. Ctrs. 

Models for Vfarfare at Sea. 

Jfenagement Engineering Plans for Secure Cormiunications. 

Cofmunications Control and Security. 

MaltiSource Data Integration for real time Aircraft ID. 

Results of French Haclear Testing. 

Microwave Circuit Analysis and Synthesis. 

tfevy Research and Development 1982. 
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Electrooptics Tecnology. 

Field Purificatiai of Vbter of BX. 

Directs Evaluation of al Test Helmut Mounted Site. 

Air Base Survivability Workshop. 

Army Report on Chemical Warfare. 

1978 Defense Science Board Survey. 

Approach to Test Plan for Aircraft Operation in a Toxic Environment. 
Measurement of Film Cooling Effectiveness in short duratiai Wind Tunnels, 
Descriptiai of Nfejor results of Air to Ground Visual Recognition. 
Diffuser Science Board Suirmary Stuc3y in Technology. 
Infrared Decoy. 

Recent Development in High Performnce Ceramics. 
Fracture Mechanics. 

Evaluation of IXiplex Whiskered Silicai Nitride Structure. 
Condidate R&D Thrust f or R & D Initiative. 
Projectory Reconstruction Progress. 

Nfeintaining Ability Verification/Demonstration Evaluation. 
Characteristics of Nettle Vapor. 

Handbook of Insructional for Non-Deductive Testing of Nfeterials. 
Flight Safety. 
On^^ine Systems. 

NAVSTAR Global Postionii^g SystOT. 
Security , Privacy f ^tetional vulnerability 
Cryogeiic Alloys. 
Antenna. 

High Spot Project. 
La.ser Conmunication. 
Firefight 2 Model. 
Defense of ships. 
Artificial Intelligence. 
Artificial Intelligence. 

Sensitivity Analyses of Platform in Target Location. 

Reflect ivitiy Measurements witn 10.6 Micrometer Infrared Radiation. 

Lasers. 

Literature Surevey of Physical and Chenical Properties of Ak^ents. 

Military OrganiEation# Readiness and Sustainability . 

Microwave Radioactive. 

QUICK Simulation Model. 

Fiber Optic Couplers. 

Projectile Characteristics. 

Aluminum Coatings. 

Designs for Control of Projectile. 

Aluminum Alloy Technology. 

Environment Storage/ Test of NATO 7-62* 

Tactical Systems in Technology. 

Tactical Warfare. 



ERIC 



A-20 



.90 



A Prograntnable Wide Band Pseudo Random Coded. 

Radar Detection, Analysis for Map of Earth, Helicopter Flight. 

A Survey of Uiderwater Models. 

CormiLin icat ion s . 

Ship Defuse Systems AET. 

Hydrazine Detection. 

Slide Rocket Proponents. 

Nuclear propulsion. 

Superplastic f orms^Aircraft 

Documentticn of Software in the OL-192 program. 
Very High Speed Integrated Circuits. 
Journal of Insrumentation and Control. 
H.S.I.C* 

Journal of Defense Research. 
Pulsed Laser Effects. 

Chemistry Reflection change of a copper surface. 
Dive control . 

Subminiature Electronic Equipment. 
CC Intelligence. 
FFT Algorithms. 

Microstructure-Nickel Base Alloy. 
Prc^llants^ta^it ions . 
Cavitation of Punps. 

Peparation of Radiation^Adhesive Bonding. 
Aerotheanodynamics . 

Investigation of Cold Cathodes for Long Life OO^ve Guide Lasers. 

Infrared Countermeasures. 

Feasibility of Fuse for Small Caliber. 

Air Cushion Vehicles. 

Electrical Vehicles. 

Infrared Analysis of Land Combat Vehicle and Land Background. 

Effect of Laser on Army Optics. 

Pershing Missile. 

Interior Ballistics. 

Pulse Laser. 

Shaped Charges. 

Interrelationship of Rock Properties. 
Land/Mine Vferfare. 
Elastic Scattering. 
Anti-air Homing Missiles. 

IDA Report -Effect of weather Hanover FRD an electroc^itcal imaging sys, 
3 Dimensional Conputation -Target Interaction. 
Cost Analysis. 

Development of Archives for Existing Data Bases. 
USAF Avionics Plan. 



ERIC 



91 



Technology of space Based Radar. 
Analysis of Training Managerient. 
Leadership Criteria for Anny Instructors. 
Library Handbook. 

Surv^ of Perfonrance Effectiveness Measure. 

Shaped Charges. 

Missiles. 

Laser Effects. 

ADA (Computer Language) . 

Target Vulnerability-Munitions. 

Delense Repression. 

Ehvironmeital Implications of Ground Based Airborne Hydrogen Flouride Laser 

Operation. 

Radar. 

Airlift Operations. 
Library Handbook. 

AFFIT These (Bibliography), Military Retirenients. 

Mine Countermeasures. 

Electromagnetic Pulse. 

Linear Shaped Charges Mine Fields. 

SuniTBry Report on German Research on Swept Wing. 

A::oost ics. 

Soviet Irtport Controls. 
Tactical Electronics System. 
Diving . 

Foreign Suf^rt. 

Penetrating Munitions. 

Status of Research on Infrared Fiber. 

Materials ^AJlnerabilty of US-^an update. 

Danage Assessment Verification Study. 

Vacuum Voltage Breakdown as a Thermal Instability of the Emitting 

pultrusion. 
Circular Polarized Static RCS Patterns. 
Synthetic Radar. 
Operations Analysis. 
Aerothennodynamic heating. 
HiltiH^and Antennae Systan. 
tetural liiaguage Processing. 
Comnand and Control. 
Joint A::quisition of Weapon Systens. 
Cheinical Vferfare. 
Projection Pursuits. 
Self -forging Fragment Calculation. 
(ARSV) Armored Reconnaisance Scout Vehicle. 
Magnetics. 

Fire Control System for M-1 Tank. 
Allocation of Military Resources. 
Advance in Naval Architecture. 
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White ftiosphorous. 

Tactical Nuclear War at Sea. 

Design Requirements of the ASWS Stanc3off Weapon. 
Software Cost Estinating. 

Expert System-- Newest Branchild of Computer Sciences. 
Wide Angle Drive Transmission for Hydrocoil Patrol Craft- 
Distributed Data Processing. 
Incentive Contracts. 
Contract Incentive. 

NBSR83.-23 -42 Fed. Sol. Program Users Manual. 
Test of Satellite Terminals. 
Dry oil Battery Usage. 

Develc^fnent of an AlHtetal Thick Film Cost Effective. 
MetalliEation for Solar Cells. 
Desert W&ter Problems. 

Buman Performance in Continuous Operation . 

Fuel Cells. 

Avation. 

Decision Support Systems, 

Cost Grov/th in Defense Acquisition. 

Africa . 

High Power Millimeter Vfeve Arrpliter. 
Tank Fire Control. 

Multiple Attribute Decision Analysis model. 

Matrix Managanent. 

List of Data Sources. 

Program Managenent Information Systens. 

Foreign Air Cushion Vehicles. 

Soviet Bloc Capabilities. 

Don't Know. 

^feccine Trials. 

Casualty Study. 

A 0 Threats Scenario. 

PCOSS (Control of Space Structures) 

Assessment of Acceptability of Digital Speech Coimunication Systems. 
Is the US Pr^red for its next conflict? 

Guide for Transitioning Army Missile Systems from Development to 

Production. 
ElectrO'-Hetric Determination of Metals etc. 

Conputer Technology Forecast and Miliary Acquistions System Process. 

Maintenance and Diagnostics. 

Chemical Warfare in Central Europe, 

Fault Action of Equipment. 

Heliopter Rotor Down W&sh-Tests. 

Aircraft Dynamics. 

Tabular Aids for Fitting widral movement esculant- 

A Simulation ^!odel for Reliability/ Availability/ K tintainability (RAMA) of 
AiTphibian Assault landing craft- 
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Chemical Stinulant. 

US/Soviet Strategic Balance in '80- Can We Meet the Challenge 

Base Contracting Procedural (2 vols.). 

Operations Research -0 Career Field. 

Roman Dimensions-Protective Clothing. 

Instructional Technologoy. 

Truck Driving. 

Imagery and Learning. 

Iteval- Blue-Green Single Pole Downlink Propagation Model. 
Corrosion - Prevention in Tactical Vehicles. 
Logistics Support at the Corps level. 
Potential for Conflict in Latin America. 
Jet Flow Phenomena. 
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Question: 'J^t^B the g^eral subject of the last «5rch of the OTIC 
*tabase you perfoced or h.d perfo«ed for you?- (See page 



Plasticity 

Video Disk and CAI 

Group on Forecasting Methods for Defense Contractors 

Infrared Transmission of Optical Materials 

EHF comnjunications 

Moving Target Indicators 

Warsaw Pact Logistics Planning Guide 

Test Report on PLRS (Position Locating and Reporting System) 

Undersea Uarfare 

Monopults 

Security Classification Management 
Submarine Simulation 

Russian Translations on Communications Security 

Underwater Survival 

Microwave Electronics 

Command and Control Systems 

Acquisition 

Effects of Chemical Toxicants on Airplane Materials 
Absorption Phenomena 

Tig Fusion Spot Repair of Resistance of Defective Spot Uelds 

Laser Window Materials 

Fracture Mechanics 

Navigation of Course Missiles 

Non-Destructive Exam of Ceramic Materials 

Flight Safety 

Computer Communications Security 
Navigation Satellites 

Geophysical Prospecting and Site Selecting 
High Altitude Platforms 
Navy Transit System 
Command and Control 

Sensitivity Analyses of Platforms in Target Location 

Reflectivity Measurements with 10,6 Micrometer Infrared Radiation 

Technology in Weapons System 

Towee Arrays 

Integrated Optics 

Projectiles 

Shaped Charges 

Passive Sonar Signal Processing 
Rocket Propellants 

Compression of Solid Rocket Propellants 

Nuclear i 'opulsion 

Cost Productivity 

Meteorological Data Reduction 

Integrated Circuits 

Optical Electronic Instrumentation 

Miniature Electronic Equipment 

CC Intelligence 

Fluid Belt Combustor£ 

Preparation of Radiation-Adhesive Bonding 
CO2 Lasers 
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Infrared Detections 

Count ermeasures 

Maurick Warheads and Fuses 

Target Detection 

Lasers 

Defects of Airfields 
Electric Guns 
Lasers 

Tunnelling Technology 

Landmine Warfare 

Radar/Radio Frequency Detection 

Archival Mass Memory^ 

Air Warfare 

Space-bom Detection System 

Psychological Impact of Retirem<^nt on Military Personnel 

Teaching Methods and Training Devices 

On-line Searching of Bibliographic Data 

Warsaw Pact Political Vulnerability Study 

Control Systems 

Laser Effects 

ADA (Computer Language) 

Behind the Armored Debris 

Defense Repression 

Close Air Tactics 

Training of Military Instructors 

Army War College Thesis (Bibliography) 

Command Control Communications Intelligence 

Mines 

Electromagretic Pulse 
Linear Shaped Charges 
Mines in Urban Areas 
Underwater Acoustics 
Soviet Import Controls 
Air to Air Missiles 
Diving 

Radar Cross-Section Studies 
Synthetic Radar 
Natural Language Processing 
Command and Control 
Chemical Warfare 

Self-Forging Fragment Calculation 
Aluminum Armor - Spelling to Fragmentation 
Productivity 

New Vessel Series (11) - 500^T Class 

White Phosphorous 

Tactical Nuclear Warfare 

Scftware Cost Estimating 

Military Preparedness in the Middle East 

War Gaming Models 

Item Identification - Personnel Management 

Ground Shock and Soil Properties 

Mathematical Prograjmiiag 

Logistics of the West German Army 

Army Field Services 

Contract Motivation 

COt. L Fuel Availability 
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Tactical Satellite Terminals 

Development of an AJl^Metal, Thici* Film, Cost -Ef f ec tive Metallization for Solar 
Cells 

Desert Water Probleois 
Weapon System Cost Growth 
Africa 

Cold Rfigions Engineering 
Gyration 

Determine Definition of Term "Milestones" 
Productivity (Contractor) 

Cost Analysis Related to Production-Engineering 

Management Information Systems 

Combat Maintenance Capability 

Surfactants in Electrochemistry 

Software 

Job Enrichment 

Built-in Test; Automatic Test Equipment 

Logistics Policy and Planning 

Helicopter Rotor Downwash 

Rubber Materials 

Statistical Sampling 

Reliability 

Chemical Agent Simulant 
Strategic Metals (Materials) 
Operations Research - Career Field 
Protective Clothing 
Instructional Technology 
Truck Driving 
Space Radiators 
Imagery and Learning 
Publications of Naval War College 
Logistics Support 1982 
Laser Target Designations 
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APPENDIX B 



SURVEY METHODOLOGY 
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T^e OTIC survey was conducted by telephone. The questionnaires 
were administered to the respondents by OTIC interviewers using the inter- 
governmental telephone network. Ml users of OTIC were randomly sampled 
from one of two sources. For those responding to questions concerning 
technical r^>orts, bibliographic searching, nanual searching and CAB, users 
were sairpled fiosn a list of Qn-Denand Technical Report Orders for the 
period October 1981 through ;^ril 1982. A list of OTIC search requests was 
used to sajrple users of the Management Databanks; WUIS, R&DPP and IR&D. 
T^iese users represented those that requested OTIC to conduct a search for 
them as well as those that performed the search personally. 

Organizations that ordered 300 or more reports in either hard copy 
or microfiche were defined as large organizations and those with orders of 
less than 300 reports for the period were defined as small organizations 
for the purpose of this survey, in many cases the organization had more 
than one location t^iat ordered reports, if the organization had at least 
one location with more than 300 r^>orts ordered, then that and all other 
locations of that organization were defined as "large." 

Approximately 80 percent of the technical reports users were 
identified by contacting the organization library or reports center, each 
of which provided names of approximately five OTIC teciinical reports users. 
Users were then contacted, and 80 percent responded to the questionn/':ire 
administered. The libraries and information centers were sajT?)led from the 
Technical reports Qn-Demand requestor list. 

TVo main questionnaires were designed and adaptations of each were 
used for each group of nine respondents. T^ie respondent groups were 
characterized by whether they were government or contractor, small or large 
organizations, and by the parts of the OTIC system they utilized; manage- 
ment databanks, technical r^iorts, bibliographic searches, TAB, CAE and 
combinations of the above. 
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The that were sainpled from lists of technical report reques^ 

tors were requested to report which parts of the OTIC syster? they utilized. 
Based on that the interviewer would acininister the Technical Reports CXies- 
t ionna i re and one other questionnaire, ei ther a (1) Bibliographic 
searching, (2) TAB, or (3) CAB. (See ^pendix C for samples of question- 
naires.) In totals responses were received frcm 315 individuals. 

The managanent databanks respondents were queried imuch like zhe 
technical reports respondents. They were divided according to (D govern- 
ment and contractors, (2) saall and large organizations and (3) the manage- 
mer*t databank used^ In this case the respondents were also divided between 
those thaz had request Gt^ ETIC to perform a search for them and those that 
had searched ETTIC databases thenselves. The databases covered in this 
survey we^e; (1) RiiT Work Unit Informat ion Systerr (WUIS) Data Base 
describing research and technology projects currently in progress at the 
wor.-; unit level; (2) R&D Program Planning Data Base (Ri-DPP) describing 
program pl:inning a^: the project and task levels and (3) the Independent 
Researc''^ and Development {IR&D) Data Bank . The respondents onswei'ed 
questions about one specific search within one databank. The number of 
respondents to the nanagenent databank questionnaires was 13U 

S urvey InstrLir!ents 

TVo main questionnaires were designed for the survey of, with 
multiple SLibtjuest ionna ires which were combined to cover each respondent's 
usage of CTIC, 

For the sample oC on-dcirand technical reports requestors, an intro- 
ductory questionnaire was administered to identify the parts of the ETTC 
system utilized (see Exhibit A). The users were vAiether they had used 
tejhnical reportsi rranagement information reports^ TAB and TAB indexesi CAB 
or performed bibliographic searches in the last year (summer 1981 - summer 
1982) . They were asked to estiniate monthly usesi time spent on those 
inv'iividual uses^ and the percentage of uses that were classified and 
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EXHIBIT A 
IDENTIFICATION OF DTIC CSAGZ 



I, 



V, 



If yes, 
to 

then 
Il^ill 
or lY 



r roduct or Service 

'a. Technical ReporLs in 
Paper Copy 

- b , Te:hnlca i Report s in 
Mict of icV.e 

c , Hanaj;en\ent information 
Reports 

-cK TAIi iind TAB Indexes 

-e, CAB (C\irrer.t Awareness 
bibl lographles) 

,f, iiibilographlc Searching 

fl , By your organizacion 

f2. By DTIC 



Used 
Last 


In 

Year 


Yes 


_ Ho 


□ 


□ 


□ 




□ 


□ 


□ 


□ 




□ 


□ 




□ 


□ 


□ 



Average MonChly/ Time Spent Using Percent Percei^.i 



Yearly Uses 



(Specify minutes Classif Limitec 
or Hours) 



(Mo/Yr) 


(llrs/min) 


(Mo/Yr) 


(Urs/miu) 


(Mo/Yr) 


(Hrs/miii) 


(Mo/Yr) 


(Hrs/mln) 


(Mo/Vr) 


(Hry/min) 


(Mo/Yr) 


(Hry/mln) 


(Mo/Yr) 


(Hrs/mln) 


(Mo/Yr) 


(Hrs/mln) 
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lunited (of technical reports only). The OTIC product and service uses 
reported acted as a cue fcr interviewers in determining the subqiief^t ion- 
naires to be acininistered. 

(>,iestion 2 of the introductory questicrmaire reported other sources 
of infonraticn othei" than OTIC utilized by the respondent and aided in 
identifying the proportion of use coirpared to use of DTIC (see Exhibit B) . 
CXiestion 3 defined the work role oji: the respondents r which was irnportant 
Iriter ill the analysis of the survey results. 

The subquestionnaiies focused on the respondeit^s last use of OTIC 
product or services and the characteristics ^d value of the information 
gathered through that request. In addition! the users were queried con- 
cerning the amount of time spent reading or searchingr the source of physi- 
cal accesSf and how the data gathered were applied to projects or tasks. 

The possible combinations of the questionnaire were numerous and 
acted to define the respondents by their types of use of OTIC infoniBtion. 
Hiey also helped to link the types of use to the user's perceptions of the 
value of the data gathered using the OTIC products and services. 
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EXHIBIT B 

OTIIHR SnURCKS OF INFORMjMION AND WORK ROLES IDENTIFICATION 



We ar*» also Interested in your use of other sources of information, 
r'ic3se indicate your average monthlv use and the time spent on the 
ID Ilowing; 

Average 
Monthly 

Uses_ Jiae Sptent Using^Ho^^^ 

a . \'cn-DOD technical reports min^ • -^hrs • 

b- Journal articles ^mins-Zhrs* 



Other naterials ^mirs./hrs. 



3, 'vliat is your primar>* work role vithin the organlriatit'^nj thnt is, vhnt do 
yo'j spt^nd the lar^^est proportion of ;"our tiz^e dolnp? 

a- Manag^nent | ' "! 

b. Teaching \ _ 

c. Research ( 1 

d. Other ! 1 
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APPEfDIX C 
DTIC SURVEY QUESTIONNAIRES 
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interviewer name 
date of interview: 



naTDe : 

organization: 



phone : ( 

V, M-VNAGEMENT DATA BANK SURVEY 



According to our records^ 

1) You searched 

2) DTle -searched for you 



from 

a) th& R£,T Work Unit Data Bank 

b) che RitD Program Planning Data Bank 

c) the Independent R^D Data Bank 

f within the last month. Could you please give us the name and telephon 
numor of individuals for whom requests were made? 



C-l 
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INTRODUCTIOK 



Hello, Eiay I speak to 



(Name of person called) 



Hello, rhis is (na::ie of interviewer ) , I ani with the D^partnient of Defense 
Techr,ical Inforriation Center, We are making some inquiries in order to 
gather data concerning distribution of our products and services libraries and 
information centers ^ 

I would appreciate it if you would take a moment to answer several questions 
concerning this inquiry. Any information will be l^ept in strictest confidence 
(Defense Departnent ^3507) • 



We understand that you recently received output from a search of 

a) RSD Program Planning Data Bank (R&DPP) 

b) R£.T Work Unit Information System (WUIS) Data Bank 

or c) Independent Research and Development (IRfiiD) Data Bank 
The questions I am going to ask you are about that st^arch output* 

J. For what purpose did you request the search? 



2, Were the results useful for that purpose? Yes 



No 



Why: 



3, To the best of your knowledge, is the information you obtained; 



complete 



Yes 



No 



Don't Know 



A, Did you pass the information you obtained on to anyone else? 



Yes 



No 



How many people? 
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Has the information you obtained from the data bant saved you and/or your 
coworkers an\ time? 



Yes 



□ 



No 



□ 



Don't tCnow □ 



If we consider the average hourly vagas of your staff workers times the hours 
of searcli time, what is the approxiroaCe dQ_ljjiT y^jlue of the time you and/or 
your co-workers saved? (Insert zero **0*' if none,) 



What^ if any, were the dollar savings for other things such as 
equipment and supply costs? (insert zero "0" if none,) 



Could you describe how it saved you time? 



(Probe: Vher would you have had to do if you hadn't read that particular 
informat: on?) 

About hov many times in the pac^t have you used DTlC's management data bases? 



How frequently do you expect to use the system in the future? 




ORC: 



OTIC VSER gUESTlOKNAlRE 



1 . 



v. 



if yes 



IKTRODUCTIQ^ 



TITLE: 
rHONK;" 



Hello* m-ay I speak to 



(N*arne of person called) 



Hello* chls is <nanie of interviewer) . I am vlch che Deparctnenc of Defense 
Technical Informac ion Center* We are making soiDe inquiries in order co 
gather data coticerning distribution of our products and services to libraries and 
Infornj t ion centers * 

1 would appreciate it if you would tak*' a monenc co ansuer several questions 
''oncernitig this inquiry. Any inforaation will be kept in strictest confidence 
(Defense Deparcnent ^'3507)» 

Ve are interested In finding out about your use of defense related publications* 
particulatly those distributed by DOD*£ Technical Information Center* I am 
going CO read you a list of six products and services provided by DTIC* For 
each one* I would like to know: (1) whether you have used the product or service 
in the last year* (2) if yeSt how tnanv times you use the product monthly or yearlv* 
(3) the nutrbor of hours you spend using the product or service in an average 
Tiontb or year* and (i)vhat proportion of thJ^ use Is relative to classified 
materials and uhat proportion is relative to litatted materials. (FOR CATEGORIES a.-b.) 



Used in 
Last Year 

Produce or Service- Ves No 



. Technical Reports in j j j j 

Pap.^r Copy j \ j \ 

. Technical Reports In \~] (ZZ) 
Microfiche 

Manai^einent Information j I [ [ 

Repcrts I I I ' 

fd. TAB and TAB Indexes 

CAB (Current Avareness j 1 1 1 

Bibliographies) I— I 1—1 

Lf. Bibliographic Searching ]__^ I I 

fl. By your organization 1 I f ~ I 

t:. Bv DTIC CD dJ 



Average Monthly/ Tirae Spent Using Percent Ferceric 
Yearly Vses (Specify minutes Classif lit:iited 
(Sjjec if v) or Hours) 



(Mo/Yr) 


(Hrs/mln) 


<Mo/Yr) 


(Hrs/niin) 


(Mo/Yr) 


(Hrs/mln) 


(Mo/Yr> 


(Hrs/min) 


(Ho/Yr> 


(Hrs ^min) 


(Mo/Yr) 


(Hrs/mln) 


(Ho/Yr) 


tHrs/min J 


(Mo/Yr) 


(Hrs/cln) 



(IF RESPON^DEN^T A^SSUTRS SO Jo {b)* (d)* (e), and (f), STOP AFTER Question 3.) 



Can You tell us hov »any researchers tsclenttsts and engineers) are in your 
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We are also interested in your use of other sources of information 
Please indicate your average monthly use and the time spent on the 
following; 

Average 
Monthly 
Uses 



Non-DOD technical reports 
Journal articles 
Other materials 



Time Spe nt Using/Month 

^mins • /hrs, 

^mins • /hrs • 

mins • /hrs , 



What is your primar>^ work role within the organl2,ation, that is, \;hnt do 
yoj spend the lar^eftt Proportion of yoLV tine dolnp? 



a. 



Management 
Teaching 
Rv:^earch 
Other 



L 



# 
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1. TECHNICAL REPORT QUESTIONNAIRE 



1. We are interested in your use of reports provided by DTIC. Can you tell us 
what report you read last? 

Title: 

(SPECIFY: FICHE or PAPER COPV AND CLASSIFIED raCLASSIFlKD/LlMITED__ 

or raCLASSIFlED/LT^LlMITED ) 

2. How did you initially find out about this technical report? 

It was routed to me 1 

From another person (i.e.^ a colleague) 2 

Cited in another report or journal article 3 

From 3n announcement publication which include: '* 

STAR TAB GRAfifl OTHER (name) 

From Cab (Current Awareness Bibliography) 5 

From 3 Bibliographic Search 6 

Done by your organization 

Done by DTIC 

Other (describe) . . . 7 

3. From which source did you obtain physical access to this technical report? 

It was sent to me via standard distribution 1 

It wa'^ KCTit to tny Li brary/t.-^rgani z.^ t i on via standard 

distribut ior. 2 

1 ordered it 3 

>fy L3 brary/ organ izat Lon ordorod it 4 

Other - . . 5 

4. How thoroughly did you read this report? 

With great care 1 

With attention to the main points 2 

Just to get the idea 3 

5. To how many others did you recommend this report? 
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In which of the following ways have you used, or do you pi n to use, the last 
report you road? 



Yes 



To apply its findings to a current project.... 
To apply its methodology to a current project. 
In preparation of a research proposal 



In preparation of an article, book, review, or 
report 

As a citation in a journal article, bi^ok, review 
or report 

For professional development , current awareness , 
or general interest 

In preparation of a lecture or presentation 



For the planning, budgeting and Tnanagement of 
research 



No use 

Other (describe) 



No 
2 

2 
2 



Don^ t Know 
3 
3 
3 



3 

3 

3 
3 

3 
3 
3 



7a< Has reading these reports saved you and/or your co-workers any time on any current 
task or project? 



Yes 

No , 

Don* L Know, 



7b< What did you and your staff save a.s a result of that reading; 



If you con?;ider the amount of .qtaff time devoted to searching for the.so 

material?; and the averap,e salary of the staff, what is the approximate 

dol laj^ value of the time you and/or your co-workers saved? 

(Insert 2ero ^*0" If none.) 



What, if any, were the dollar ?iaving.s for other thing.s such 
as equipment and supply co.sts? (Insert zero "O** if none.) 



10. Could vou describe how it saved vou time? 



(Prober What would you have had to do if you hadn't read that parrfculnr 
Inf ormat i on?) 
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11. MVXUAL SE:\RCH QUEbTlON^NAlKi: 



1< When was the last time you used TAB? 



V]\oti. was It for: Browsingp Problem^relat ed searchn 

Did yt-^'J personally perforin the search^ or was it pertormed for you? 
Pt^r formed search per.^onal ]y 1 
Search was pt^rformed by someont' else 1 



3< How much time did you sp*5nd on "he search? 
^a- How many iteins were identified? 



b< How many were: classified | | 

unclassif ied/ limited j ( 

unclassified/unlimited | | 

3< To how many of these item.s did you physically obtain access (or intend to 
obtain accefc^s) ? 



6a, How many itcn^t^ did you read (scan beyond tiie title page)? 

b< Hciw many of theso items read wore: 

Classi f ied _^ 
Unciassi f ied/Limi t ed 
Unclassi f ied/ Unlimited _^ 

7a< ^^as reading these reports saved you and/or your co-workers any time on any 
current task or project? 

Yes 1 

No 2 

Don * t Know 3 

7b- Wlirit did your staff 5?avo as a re^iul L of that reading? 
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6, Considering only direct sslaries. what is the approximate dollar 
val ue of the time you and/or your co-workers saved? (Insert 
zero "C** if none, ) 



U^nat. if any, were the dollar savings for ether things such 
as equipment and supply costs? (Inse^'t zero ^'0*' if none,) 



10, C0L:iG you GC:scribe how it saved you time? 



(Probe: V^nat would you ha\'e had to do if you hadn't road tllfet particular 
iTi fo rrr^z t ion? ) 
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111. B1BLI0GR.\PHIC SEARCH QUESTIONNAIRE 



1. What was the general subject of the last search of the DTIC data base you 
performed or had performed for you? 

2. Uliat nethod of searching was used' 

Scarcli was perfomed 

by your organization 

DTIC-Provided 

Searching 

3. Did you personally perform the search* or was it performed for you? 

Performed search personally 

Search was performed by someone else By WhoD? 

4. How much time did you spend: 

Requesting and/or conducting search 

Reviewing search output 

5. How many citations did you receive as output fiom the search? 

6a. How many citations were relevant? 

6b . How manv were : class i fi ed unc J ass if ied/liuited unclassi f it^d/unl imi ted 

7. To how many of these items did you actually obtain access? 
8a. How many items did you read? 
b.How many items read were; 
Classified 

L'uciassi f ied/Limited 

Unclassi f ied/Unlimlted 

9. Has reading these reports saved you and/or your co-workers any time on any 
current task or project? 

Yes 1 

No 2 

Don't Know 3 
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we consider average salary of your staff workers times the amount of 
tine thev would spend searching, what is the apprcxiniate d^.^ lar value 
01 the time you and/or your co-workers saved? (insert zero "0" if none,) 



What, if any, were the dollar savings for other things such as 
equipment and supply co3ts^: (insert zero '*0 if none.) 



Couid you describe how it saved you time? 



(Probe: Wiat would you have had to do if you hadn't read that particular 

int orma t ior ? ) 



IV, CURRENT AWARENESS EIELIOGR.\PHY Ql^RlSTIONNAIRE 
When did you last roview :i currcnn awareness bibliography? 



Ilnw mATiy were: clas.^i Cied unc la.ssi f ied/Limi t ed unc la?; .si f ied/unllml ted 

How many relevant citations did you identify? 



How many reports did you request or plan to get? 

How many items did you read? 

How many of the reports requested were: 

Classified 

Unclassif ied/Li3iited 

Unclassi fled /Unl imi ted 

Has reading these reports saved you and/or your co-workers any time on any 
current task or project? 



Yes 1 

No 2 

Don' t Know 3 



Lf we consider tiie average staff worker's salary and the time saved x number 
of staff* what is the approximate dollar value of the time you and/or your 
co-wotkers saved? (Insert zero "0^' if none<) 

$ 



W^aCt if any* were the dollar savings for other things such as equipment 
and supply costs? (insert zero "0'' if none<) 



Could you describe how it saved you time? 



(Probe: What would you have had to do if you hadn't read this particular 
information?) 
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